Male arousal by breasts is a fairly recent phenomenon.

Covering the genitalia probably has more to do with their vulnerability than the potential for arousal leading to rape.