You seem to be upset simply because I don't think sled pulls offer much benefit for anyone aside from those competing in events that utilize similar. Obviously, if someone is very muscular, most will assume that whatever he is doing is working for him. If someone is small, they will not get that same benefit of the doubt and they truly shouldn't in my opinion.
As far as the "lean" comment, that would depend again on how muscular someone was. If some guy had 16% body fat but arms over 20", most people are not going to assume he is wasting his time in the gym. I don't know what gyms you go to, but I seriously doubt this is taking place in any of them.