You say absent a medical exam, there's no way to know someone's sex. I would say absent drastic medical interventions, the number of people who you would see as "passing" for the opposite sex is vanishingly small.

I have to say, after seeing a video of Blaire White walking down the street, I found his female presentation less convincing (previously seeing videos showing head/shoulders, like an ancient Roman bust). I thought the placement and movement of the boobs was particularly troubling, and the walk, definitely male.

In the instance of Buck Angel, I think the bald head/beard combo plus mastectomy plus obesity plus the gravelly voice (probably a mixture of testosterone changes to vocal cords and some voice training) is more convincingly male than Blaire's presentation as female. Both Buck & Blaire are products of surgical & hormonal interventions that have only been available in recent years. (Btw, Buck never had "bottom surgery" and made a lot of money inserting objects in porn videos).

There's a you tube video called, "I showed It's Pat to Nonbinary People" which you might find interesting.

Having been a teen when SNL first aired, I have to say that the way people took the Pat character was not as an icon of androgyny, but as a *massive* weirdo. Julia Sweeney said she invented the character because she was unsuccessful at portraying a male character, a guy from the accounting office where she was working, whose mannerisms she was trying to use comedically.