I noticed something like this back in my high school wrestling days.
In the finals, at tournaments wrestlers in the middle classes (138, 145, 152) tended to be sort of "average" in height and strength, but very skilled at wrestling.
At the very low end (105-112) and the very high end (189-220) you'd start to see more "extreme" heights or strength, or weird combinations. The 4ft 11in 105 pounder who manhandles everyone, or the 6'4" HW who is just way more athletic than the other heavy weights. These guys were still skilled, but their body type, or better than average strength kinda over whelmed their opponents.
And at the extreme ends, the exhibition weight classes (98 or 265+) you could almost tell who was gonna be best by looking at the weigh-ins. The best guys at the extreme limits were just way too much physically for anyone else to handle, regardless of their skill. They stuck out from everyone else in their weight class because of the extreme height, or really strong looking build.
Anyway, sort answer, yes. I think that above average strength or height or speed, or any other skill is more of an advantage at the lowest, or the heaviest weights than in the middle weights.