There may be more factors than these, but I think relevant things include:
Head size. A bigger head results in the same body looking smaller, and vice-versa.
Rib cage size: A big rib cage and larger lungs results in larger volume for same weight (lung tissue takes up a lot of space for a given amount of weight.)
Hip size: An amount of weight in the hip structure put instead into the same number of pounds of muscle elsewhere gives an appearance of larger size. If for the same weight, height, bodyfat and other factors someone has a heavier hip structure, he’ll look smaller overall.
Bone thickness: Heavier bone structure, for same bodyweight and bodyfat, results in smaller appearance. Not, of course, that small bone size tends to be a contributor for guys who are really big, but having relatively heavy bones for their weight may hurt the size appearance of a number of guys who are of a smaller-looking size.
For example, way back early in training when I was 158 lb and not ripped, I was underwater weighed at 6% bf. (As personal opinion, true bodyfat was probably 10%.) The best explanation for that is relatively heavy bones for my weight, despite a small frame.
Personally, I manage to combine all four of these features (or perhaps now only three: the last is now open to question, as perhaps at my current weight the bone weight might be proportional) and I look at least 20 lb lighter than actual weight. Which of course is an awful trait for bb’ing.