Speaking from being an amateur boxer, gaining weight helped my performance, not hindered it. I got stronger and was no longer bullied/shoved around the ring while sparring heavier guys in my gym, and my speed was exactly the same if not better. Power definitely increased.
The only thing that I can say maybe got hurt, was my endurance. But this connection is weak at best, and I'm fighting as a LHW instead of a junior middleweight now, so the pace and physicality is different anyway.