Federal offices (i.e. post offices and banks) are closed on specific days; but other businesses decide for themselves what they want and can afford to provide. There is no federal law requiring you to give any holiday off -- either with or without pay.

Most employers prioritize the "usual" holidays and designate some of those to be allowed taken off with pay in order to attract and retain good workers. Some employers also offer one or two floating holidays to allow workers to select days personally important to them to take off with pay (i.e, birthdays, religious holiday, etc.)