That is a good point. I remember my old mentor tell me something about big NFL players. He said that they were "oversized normal humans" meaning that they kept the same proportions as shorter people but where just on a larger frame. This is different than "tall people" who seem "elongated". The later do not seem as large because proportionally their width and thickness isn't in line with their length. "Tall/elongated people" also seem to have less coordination and body control which means that they sometimes look clumsy and don't normally make good overall athletes while "oversized normal humans" can have the same agility and body control as shorter folks.
I believe that the "oversized normal humans" have the potential to be much stronger than "normal people"... look at the top strongmen for example... Hafthor Bjornsson and Brian Shaw for example. Both are 6'9" and are now over 400lbs. But they used to be much lighter during their basketball day but were still strong. Once they added muscle to their frame they became the strongest in the world.
Another example is Wilt Chamberlain who could deadlift humongeous weights (although tall people with very long arms tend to be naturally good deadlifters). Lebron James and many other NBA players also fall in that category.
But yeah, I believe there is a difference between "large people" and "tall people". The former tend to have the potential to be stronger and also have much better coordination than "tall people".