You have too many threads for anyone to have insight into your case.
If SERM dose is high, a common problem with bro-science and many doctors, LH receptor desensitization could result. Then is dose is increased, it gets worse.
SERMs are not going to work with primary hypogonadism; however scrotum should be independent of that.
When on a SERM, LH/FSH labs can be very informative.