This article has several coaches talking about when and why/why not to wear a belt.
Not sure what you mean by "bodybuilding sets", but "going as heavy as you can and ignoring time under tension" is very much not the point of bodybuilding-focused training.
Based on the people you see doing both, what are the guys in better shape doing? Try doing that.
That's two goals, and the most efficient way to either one is contradictory to the other. Prioritize.