I think I can bring an unique viewpoint to this discussion in that I never lifted weights until I was 36. Currently, I'm 39. I ran CC, swam, and did track in HS, and competed in triathlons during and after college. I was really fast and had great endurance, but also had the metabolism of a hummingbird and graduated HS at 6', 140 lbs.
So, 3 years ago I stepped into the weight room 6', 170 lbs knowing nothing but smart enough to have bought and read a copy of Starting Strength. Some of my discoveries after 2.5 years of lifting:
1) GOMAD makes for a fat 36 year old. Eating excessive calories does not build more muscle, it adds weight to my gut and ass.
2) Gaining excessive weight means losing it, and body recomps at age 39 are very difficult. You really can't serve 2 masters the way you can when your younger.
3) Losing weight is much more difficult whether it be for metabolic reasons, hormonal reasons, the daily grind, etc.
4) Strength gains are made slowly, size gains are made even more slowly. It just seems that 39 year old bodies do not want to create new muscle tissue. Guys in the gym remark that I'm strong for my size. When I first pulled 405, I measured my biceps cold, unflexed. They were 13.25". I can front squat 265 but my thighs are 22.5".
5) Recuperation is a huge limiting factor. When I was 20, I could get wasted, wake up the next morning and start firing up all over again. Now, if I get anything more than a strong buzz, I'm sluggish for a day and a half. After a heavy DL day, my upper body lifting the next day suffers. I'm just generally dragging ass. I've learned to avoid lifting until failure on too many sets, it seems that grinding out reps sets me back days. I wholeheartedly agree with "Stimulate, don't annihilate."
6) Injuries and fatigue accumulate in a way that they never did when I was younger. In my case, I'm very conscious of making sure I do my stretching and mobility work. I heal more slowly than I did when I was in my 20s and an injury will set me back farther.
Obviously, these are my experiences, ymmv. But I agree with most of the other posters that the guys who are big in their 30s got that way after 10 years of dedication and persistence. They didn't just blow up when they turned 34.