If you mean Men's Physique, and not Classic Physique, for reference @BrickHead (on the far right) placed second in a local, natural/very-not-IFBB show at 5'10" and a lean 170-something here:
The guy in the center weighed 155-160-something at what appears to be roundabout the same height and won that show.
Again, that's nowhere near IFBB level. Pics of you would help, like we've asked from you in other threads, but at your height and a very soft 162 pounds, you have a very long way to go before considering anything like the NPC/IFBB.
Considering that bodyfat is a measurement too, I disagree with your assessment. Keep building your muscular base for now.
EDIT: Did a little more checking to give some perspective. Ismael Martinez won his IFBB pro card a few days ago. About 5'6" and a very lean 170-180.