You should assess whether or not you have adequate core stability to be "only training your core with squats and deadlifts". I think the answer is - you could get away with training core only with the squat and deadlift ONLY if you already have adequate core strength to be perfoming those movements correctly.
What good will it do to be trying to squat and deadlift for core strength if you lack the core strength to maintain a neutral spine on the deadlift and squat? Deadlifting in complete lumbar flexion isn't going to build any core strength. You'll only make matters worse. I'd say almost everyone would benefit from some direct core training.
While you're at it, why don't you train your core the way it's designed to work? The function of the anterior core isn't just flexion - otherwise you'd probably have two big long hamstrings running up your ventral side. I (and a lot of good strength coaches) think that the main function of the core is to stabilize the lumbar spine. Why not do some isometric core exercises that actually function to stabilize your spine? Planks and rollouts are good examples.
Go read some Mike Robertson, Michael Boyle, and Stuart McGill articles - they will help you a lot.