OP, why worry about what other people think? Granted, AAS is guaranteed to bring you some unwanted attention, and success in the gym-whether it's AAS related or 'natural', irrespective of hard work-will get you plenty of vitriolic comments, you can bet dollars for donuts on that.
I used to be worried about 'looking' like an AAS user, obviously no one, or I would imagine most people don't want their families to know-my mum thinks steroids automatically make men raging raping psychopaths. I use AAS, ad whilst i certainly don't look like a bodybuilder my peers have seen me grow from (an underweight) 155LB to (at my biggest and leanest) 230LB, which at 6.2 isn't particularly impressive but is certainly a significant change, which has brought me plenty of vitriolic comments from people who I would have expected more positive feedback from. Understand: when people see you progressing in life-at anything-they become jealous, of your success, your work ethic, your self discipline-whatever. And just to clear up any concerns or misunderstandings, no one 'gets too big' by accident, when you do your first cycle, you'll be impressed with the results if you do everything your supposed to do, but you'll also realize how hard the pro's need to work and what they have to do to get to where they are, like I said it's not an accident.
The point I want to make, is that whilst you shouldn't use AAS unless 'your ready' (Disclaimer: Read steroid newbie thread and other stickies before using any AAS, then post proposed cycle for advice and critique) you sound like you want to use, and if you do, it must be your choice, no one else's. So, why not do something you want to do to keep your workout partner happy? Start thinking for yourself, if he changed his attitude towards steroids, would you be like 'wow, great, I'm allowed to use now!'