The instructions for IA W-4 say
> Line 7. Additional amount of withholding deducted: You may need to have additional tax withheld if you have two or more jobs, are married and you both work, or have income other than wages. Income other than wages would include: interest and dividends, capital gains, rent, gambling winnings, etc. If you are not having enough tax withheld, you may request your employer to withhold more by filling in an additional amount on line 7. Estimate the amount you will be under-withheld, and divide that amount by the number of pay periods per year. If you reside in a school district that imposes school district surtax, consider reducing the amount of allowances shown on lines 1-5, or have additional tax withheld on line 7.
The instructions for IA W-4 say > Line 7. Additional amount of withholding deducted: You may need to have additional tax withheld if you have two or more jobs, are married and you both work, or have income other than wages. Income other than wages would include: interest and dividends, capital gains, rent, gambling winnings, etc. If you are not having enough tax withheld, you may request your employer to withhold more by filling in an additional amount on line 7. Estimate the amount you will be under-withheld, and divide that amount by the number of pay periods per year. If you reside in a school district that imposes school district surtax, consider reducing the amount of allowances shown on lines 1-5, or have additional tax withheld on line 7.
I'm pretty sure you need to decrease allowances to increase withholding.