He's a lightweight fighter though. Even now, he only doesn't go back down because of Frankie beating him twice (and him being at the tail end of a career).
Still we could go point by point:
-Beat Hughes (yeah that was the shit, especially considering how dominant Hughes was).
-Went on to fight abroad, highlighted by his fight with Lyoto in which he more than held his own. Granted, Lyoto went on to drink more piss and level up considerably, but he did beat Franklin/Bonnar around that time.
-GSP beat him, but that fight was the definition of LnP. It was a testament to Georges heart, and Penn's crappy gas tank at the time (and his whining).
-Hughes beat him... after losing two rounds. I remember watching that and thinking Hughes was getting clowned. Even when he got the takedown, he couldn't do anything with it because of BJ's defense. BJ would then proceed to separate a rib and 'gas' suddenly in the 3rd. That explanation is generally accepted because no one had seen Penn gas so suddenly before, even back then.
-GSP PWNed him. Even then, remember, Penn was coming up in weight, didn't do it well if his training videos and subsequent condition were anything to go by. Despite this, Georges has yet to fight an opponent, since the Penn fight, with the same vigor he attacked BJ with. That says something. He knew Penn was dangerous and it brought the best out of him. GSP, post-Serra (hell even before), was never sharper, or more ferocious than in that fight in my opinion.
-Got a draw against the #2 at WW. Fitch is boring to watch 90% of the time, but remember, in him you have one of, if not the best all round grappler at WW. He's a legit black belt, and is strong as an ox no matter his diet. BJ took him down and held his own on the ground, not standing like everyone expected. Hell, I gave him the first two.
All that said, I feel you with the Penn nuthugging. Still there's a reason he's so respected. This guy was beating the best at LW in the beginning, took on all-comers at every weight, and still hangs with the best today, even at higher weight classes. MMA records don't mean much (look at Handy Couture), and while wins are important, they aren't as important to fans as fighting the best.
Oh, and Frankie lost the first fight