If that is the case, you must pay him time and one-half for hours worked over 40 in that workweek. In some states, it is over 8 hours in a day -- but you're asking about a holiday, so this is my response to that particular question.
You can give a different day as the full-time employee's paid "holiday", if you choose to do that.
As it is highly doubtful that the part-time employee would be working in excess of 40 hours in a workweek, the answer is simply that you do not need to pay her at any rate other than her regular hourly rate.
The exception to what I have stated above is if your employee manual promises something different than the law requires. For example, if it states that employees required to work on a company holiday are paid at a premium rate for those hours. In that case, the stated company practice should be followed.
The key to this kind of circumstance is clear communication. Advance notice of requiring someone to work on a holiday is good practice. Often, just making sure that people understand the situation ahead of time can forestall negative situations from arising.