I feel like what people are saying is completely valid - some people simply feel better without a belt. But a few things to think about :
First of all, 10mm powerlifting belts aren't the only belts out there haha. There are olympic lifting belts that are significantly less diesel than your typical powerlifting belt. Maybe that would be a good compromise for you. You might be able to get a little more stability than beltless, but it wouldn't be so uncomfortable.
Also - I think there is something to be said about just sucking it up as well haha. As some of the other posters pointed out, some people just don't pull as well with a belt. But from the way you described it, it doesn't sound like the belt is actually preventing you from getting into an ideal starting position, it sounds like it's just uncomfortable and you let that distract you and that is messing up your setup. Obviously I can't know this without watching you. But I can say from my experience that wearing a belt HURTS sometimes. Last time I pulled a max deadlift with the belt, I had bruises on my stomach for like 4 days haha. Maybe you just need to try putting the belt on, and setting up fast + pulling. Try and keep everything exactly the same as it would be without the belt. Don't let the discomfort throw you off mentally. If you give that a shot and it still feels shitty (and you feel confident it's not just you being whiney), then you probably are just better off without the belt, or trying a lighter belt out for size.