read good calories bad calories by gary taubes, and then come to the realization that the government diet recommendations are way off, and high total cholesterol has almost nothing to do with heart disease. also note that you're better off drinking milk than eating cheap carbs.
don't listen, they've likely been indoctrinated with the idea that diets should be high in carbs and low fat, when that is in fact bad advice.
but read the book. here's a link:
Yeah I realize that the common view on nutrition is off. Having a family friend who's a doctor who preaches about how bad milk is doesn't help either. Oh and it isn't fat in general, it's milk fat specifically they talk about.