Whoever told you in order to gain muscle, you have to gain fat is dead wrong. thats an old myth that unfortunately alot of guys here still beleive in. I have known so many guys that not only didn't gain any fat and gained alot of size, but actually got leaner. yes, leaner.
Sometimes arms don't respond like other bodyparts do. You have to really experiment with many different things. what works for chest and shoulders or back, might not work for arms. maybe you need to do supersets, more reps. more volume with a little less intensity. I've seen little buck 50 dudes curling a 45 on each side. yet, they have puny arms.
learning to curl alot of weight is easy, using leverage, momentum, using other muscle groups to help out. thats why these guys have big curling weight with small little arms. unless you isolate the muscle, and figure out ways to really hit it, you will never get the arms to grow.