I bounced for a couple years, and wasn't a big guy at all: I'm about 6'2" and at the time my weight went from 185 to 200 lbs. Granted, I was damned cut, so that was pretty straight-up muscle mass, but that's still not huge. Not by a long shot.
Thing is, most people get this idea stuck that bouncing is about size, strength, or kicking the hell out of people. Or all three. It's not.
It's more about awareness, confidence, and a cool head. Seeing the pattern of the trouble before it starts, intervening, talking people down from an agitated state, and keeping the peace. If it comes to a fight, it's not about being Bruce Lee. Most nightclubs pay shit wages, and it's not worth a punch in the mouth for that cash. So bouncers gang up, grab the jackass, and eject them with minimum fuss.
Sure, fights happen. And yes, you should know how to handle yourself in one. No, you're not expected to be an action hero. Any good manager will not hire someone who's going into the job with something to prove.
Granted, at the club I worked at, I was one of the smaller guys. One dude that we nicknamed "House" was only 5'10", but tipped the scales at over 300 lbs. Strongest bastard I've ever met. Two other guys went on to compete in Strongman competitions, and several were football players.
Whole crew respected each other though, regardless of size. What matters most is having your shit in order, and being a stand-up kind of guy.