I'm 61 years old, and I would say that before the past decade, I encountered only one individual whose sex I initially misread (he was a very petite gay guy with a high pitched voice, and after about 5 seconds I realized my mistake). In recent years, I've occasionally seen people whose sex isn't immediately apparent, but I think there are 2 main causes of this: 1. there are people who go to great lengths to conceal their actual biological sex, even using drastic surgery, 2. Obesity seems to blur secondary sex characteristics (Watching "My 600lb Life" I noticed some of the men looked more feminine when very fat, sometimes looking more masculine after substantial weight loss, and some of the women looked less feminine at their highest weights, and more feminine with weight loss).

Here's an example of how robust biological sex is: Jazz Jennings. This poor guy was identified as a trans child at around age 2 or 3, with a doctor's appointment confirming his trans status at age 3. He started puberty blockers at age 11, so that when, at 17, he had his "bottom surgery" there wasn't enough tissue to do the standard penile inversion, and other tissue from his abdomen was used (and there were complications requiring at least 3 subsequent surgeries). Jazz never went through male puberty and had every method available to give him a feminine appearance (cost was no barrier, and he had access to the most prominent surgeons in the field). Yet Jazz still looks male. One example of a male feature that he exhibits is "mid face" - the proportions between bottom of the nose and top of the upper lip, the size and shape of cheekbones and distance between eyebrows and eyes are different in males and females, and no surgery (and going by Jazz, no hormone alterations) can give a male a female mid face.