I don't think the distinction here is so subtle, but even forcing someone to serve people he doesn't want to is quite the infringement on liberty. Imagine if the law said you had to invite all of your neighbors to your backyard barbecue, rather than just the ones in your "identity group". Legislators came up with an end-run by declaring businesses "public accommodations", which makes them sound as if The Public has rights to them, when really they belong to the people who invested their time and effort to create them.

These laws made sense as extraordinary measures to deal with the extraordinary problem of continuing, severe racism, but now they're not insisting that a black man be able to buy a suit, they're insisting that you not hurt someone's feelings by disagreeing with his choices or beliefs.