Everyone in the womb begins with female plumbing, and at some early point each male is bathed in androgen, stimulating physical and neurological changes. The clitoris is homologous to the penis, the brain develops differently.

When there are mixed-gender fraternal twins the female fetus can get a touch of this and she will potentially appear more masculine than most girls but not enough to be mistaken for a boy.

However there is a rare condition called Androgen Insensitivity Syndrome in which the male fetus doesn't react as he should and who comes out even so feminine as to be mistaken for a girl. It takes a chromosome analysis to be certain. I've seen a few of these as adults; they are feminine in a way that no gay man can emulate, with subcutaneous fat and without thickened facial bones.