Exactly. I would say focusing on sperm counts and menstrual cycles is a much more reliable indicator of what may be going on than childbirth data. What seems to be left out of this discussion is that unlike animals, humans can and do make decisions regarding childbearing. So birth rates ALONE are meaningless when talking about humans.

Let's take a herd of deer, for example. Deer do not make conscious decisions about whether to have fawns or not. It's a matter of instinct and hormones. When the urge strikes them, they mate, and if everything is working correctly, fawns will be born in the spring. They do not deliberately practice nonprocreative sex. So if you have a herd of deer and all of a sudden you notice that there don't seem to be as many fawns as you would normally expect to see in a herd that size, and this trend persists, something is happening. The deer themselves are not choosing this. This is also why we spay and neuter cats and dogs. Because their instincts say "reproduce" and they can't say no to that.

We humans on the other hand, can and do say no to the urge to reproduce for whatever reasons we want. And especially now that we have more effective forms of contraception. The pill by its very nature impairs fertility. Meaning, that if there are any other underlying fertility issues, they won't come to light unless and until there is an attempt to get pregnant--and as you pointed out, there is a peak time period for this. Take a woman who goes through her life never having sex--there comes a point where she becomes naturally and permanently infertile. Now, because she's never tried to get pregnant, we don't and can't know if she could have had children. That "system" so to speak, hasn't been tried. All we know, is that she didn't, and this counts towards a decrease in the overall childbirth rate, a very minor blip to be sure, but a decrease nonetheless.