I've made about 4 trips to Augusta and seen Tiger many times from 2 feet away. He's a pretty small guy up close, and for the record, I have nuth'in to brag about size-wise.
His waist has got to be less than 32 and chest, maybe 38. Reminds me more of a crossfit type guy. He's in shape and I wish more golfers would at least try to get more fit.
Golf is about 80% dynamic eye/hand coordination. By dynamic I mean you swing at a stationary ball from your feet up. The other 20% is strength. That's why the best women can hit their drives on average 230-250 and the biggest guys 290-330. Not a lot of deviation.
Just get stronger than average. Take up crossfit and yoga, but more importantly see a good golf instructor if you want to add some distance to your game.