The amount that an employer shells out for unemployment will depend on the sum of his payroll, his track record in keeping employees and the rates that are specific to his state. Besides, all employers should pay a federal unemployment tax that the Internal Revenue Service (IRS) funnels back to the states to help pay administrative costs for unemployment programs.