This is a large part of what I was going to say. And I'm still gonna say it, lol.
I started lifting when I got to college. None of my high school friends saw me for about 6 months. During winter break, I got comments from like EVERYONE. The roids comments, you look huge, etc. And I also went from about 125 to 150 during that period.
Over the years, the comments have progressed something like this:
friends telling me they can tell I workout, my arms look nice, etc.
people I don't know particularly well asking if I workout.
Strangers asking what I do to workout.
Strangers asking if I'm a bodybuilder (this is only in the last year).
Friends simply assuming I use steroids without asking.
I would say the answer to your question, in a general way, is that it takes a few years. I was not particularly dedicated to lifting, and didn't lift at all, for a large chunk of the last decade. If I had been, I would have gotten to the point I'm at now probably 5 years sooner.
The best indicator, to me, is when strangers approach you about lifting and stuff. I get comments almost daily now when I go out now. That's a pretty good indicator. Grocery checkout lines, restaurants, the mall, strangers walking down the street, literally anywhere. People either want to tell me how much they lift, how much they used to lift, ask me what I do, etc. It's cool and embarrassing at the same time. I still haven't figured out how to respond to some questions.