The study you are referring to is a very recent one. It was published in the May issue of "Pediatrics international" (it's a Japanese journal). I've read it myself.
It has been known for quite some time, that cow's milk consumption has some adverse effects on the reproductive system of mammals. Some studies have been done with male mice, if I remember correctly.
The reasons and mechanisms behind this are not yet entirely clear.
It can be shown that exogenous estrogen in milk is absorbed by humans, gonadotropin secretion is suppressed, followed by a decrease in testosterone secretion. This is bad news. Besides the fact that you probable want high levels of T (as a bodybuilder or strengh athlete) for it's effects on protein synthesis, and so on, this could also affect sexual maturation of prepubertal children, sperm quality, fertility, sperm quality.
To make things even worse, estrogen-sensitive cancer cells could react to the endogenous estradiol and grow.
Concentrations estrogen in milk, especially whole milk are pretty high. This is not a Japanese phenomenon, but we can see this to be the case in western countries as well.
It is true that estrogen concentrations in milk are highly correlated with the milk's fat content. Concentrations in skim milk are low and unlikely to pose a health risk for humas. This is for "normal" people, drinking 2 glases of milk a day or so.
For you as bodybuilder, drinking gallons of milk a week could be a long-term bad idea.
If you can be sure that the milk you drink is from non-pregnant cows only, go ahead drink as much as you want.
Elsewise, I really would think about lowering your milk consumption.
I am a bit on the extreme side and don't drink milk at all besides some very few hot chocolates in the winter time... I rarly eat cheese. Maybe twice a year.
Feel free to ask if you have further questions.