Yeah, my comparison sucked. I'll try again.
Anthony Ervin is a super fast sprinter. While I'm sure his swimming ability in the 1500 far surpasses 99% of the population, it is a relative weakness for him. If he focused on lowering his mile time, over time he would add seconds to his 50m and 100m times and might not win a gold medal in Rio.
The same thing applies to anyone. What do you hope to accomplish through your cardio training? Train around that.
If you are motivated by improving something you suck at, go run a 100m, a 400m, and a mile. Look at your splits to see how well you hold speed. If you fall apart at longer distances, you need more aerobic work. If you can hold the same pace for 100m that you can for 400m, or for 400m compared to 1600, you need more speed work.
Caveat: You have to factor in your power and technique, especially for the sprints.