urk it's hard to know where to start in this. First off, fight = duel and yes, most guys can duel just fine because even though it might be pretty violent, it is a limited engagement. This is different from someone who wants you dead.
(FWIW if they are fighting you, it is unlikely they want you dead. Any coroner will tell you than overwhelmingly in this country if someone wants to kill you they will shoot you or stab you.)
(Yeah I know I'm a geezer, but truth be told fighting for when you are in your early 20's is mostly about getting your place among other guys (impressing the girls too). When you get older, you get to be a bodyguard for your kids and wife, which completely changes your outlook. When you are much older, you cannot rely on physical prowess at all, which changes it again.)
One disservice that movies have done is to completely confuse people about this. Most guys really do think that slugging it out or ground fighting is what they are supposed to do.
Guys also want to think they can handle themselves, so there is also a bit of ego involved.
In any case, so what if they think that. Let them. If there is ever an issue, I'd rather deal with someone who is going to try and fight me with a bunch of techniques they don't understand than actually think about how to bring about my demise.
And I might just be full of shit...