Is your W-2 showing the correct amount of withholding? If yes, you are fine.

It's the company's responsibility to withhold your tax and make payments to the IRS weekly, biweekly or monthly. If they didn't pay the correct amount to the IRS, the company will be charged with penalty and interest, not the employee.