It's a cause for concern when you feel the symptoms.
I'm between the low and mid-300s and the symptoms are acute for me:
- Can't put on much muscle
- Losing strength despite the (small) muscle gains I can make
- When I lose weight, seems to be equally fat and muscle, rather than primarily fat.
Because of all that, it's clear that the nutrient-partitioning/anabolic effects of testosterone aren't close to optimal.
The main symptom I have is libido though: it's practically non-existent, despite being outrageous up to my mid-20s.
So if I were you and felt like my libido was normal and I wasn't sure I had muscle keeping/gaining problems, I'd just...watch to see if you were building/keeping muscle when you dial in your workouts and nutrition. If not, move to the next steps. No matter what, it won't hurt you to read through the forum stickies and everything KSman writes.