Over-responders need to take around 1/4th of the expected dose. As you are not on TRT, you should try 0.5mg anastrozole/Arimidex per week in EOD dosing. You need a 5-6 day holiday for prior dose levels to drop.
Your improved transient improvement is a glimpse of a target that rushed by.
Listed sides for Arimidex is for women who take 1mg/day who are thus driven to extremely low E2 levels. You should be getting near E2=22pg/ml.
When guys drive E2 too low, they can feel quite poorly, moody/depressed, social avoidance, worse libido ...
Read the stickies, that is the best way to gain the knowledge that you need.