Well it's good to see that you're not elitist or ignorant in any way. Guess you got lucky.
As for sprinting, I'm a recreational sprinter and there's no better site on that topic than charlie francis' site. He's the guy who coached Ben Johnson, world record breaker and caught red handed using roids twice.
It sounds like you can generate more power than him but then you lack the endurance to keep that pace up. I remember reading a post on charlie francis' site that talked about the fact that some elite olympic lifters could keep up with the elite sprinters through the first 30 meters of a race but then sputtered off. They both had the ability to generate massive power but only the sprinters had the ability to sustain it.
You stated earlier that your brother is 19 and that you're 23... meaning that you're more removed from your track days than he is. I'm pretty sure that this plays a part.
As for the upper body swinging... it is important especially during acceleration and creating momentum. Again, I'm only a recreational sprinter but I believe charlie francis' site can confirm these statements.
Being heavier means that it takes more work for you to go the same amount of speed as it does your lighter brother. It also sounds like your brother has a higher relative strength ratio and sprinters are known for having same amazing strength in their circa 160-170 lb body. Eric Cressey would be proud.
In a competition like sprinting, absolute strength takes a backseat to relative strength and the ability to convert that into power. remember that power = force x velocity. he beats you because he is able to achieve more power.
once again, charlie francis' site. As you read the plethora of information on this site... remember that sprinting is cns intensive and as such, should be done every other day. If you want to get better at sprinting, use maximal weights with low reps... I recommend reading some of Chad Waterbury's works (especially his 8x3 article). Hope this helps.
p.s. If anybody feels like critiquing my advice or just simply ripping it to shreds, feel free. I love witty banter!