If I can chime in here, ProfessorCHAOS96 said it quiet simply, you need to eat more, and from what you said consistently. Bodybuilding is one the toughest "sports" activities out there, that requires 100% dedication and consistency to succeed. I was in the same boat you were in, albeit I was always a bit heavier, after I lost the weight, I was wondering around the bodybuilding world without direction. Find a direction, in your case, it seems to be gaining weight, particularly LBM, set a goal specific to your mass gain, and follow it wholeheartedly until you achieve it; bodybuilding is all about the will to succeed.
Fuck mirrors, and weighing yourself constantly (I used to do this way to much!), don't stop until you reach whatever goal you are trying to accomplish. Eat, lift, rest, grow. Muscle growth isn't entirely based on what you do in the gym, its what you do outside the gym.
You said that you eat 3000-4500 calories a day, perhaps you have to up these calories, or be more consistent with them 3000-45000 calories is a big differential. You also said that you are eating out at buffets a lot, buffet food although good for putting on sheer weight, isn't the healthiest or most beneficial food to the bodybuilder (IMO anyway) and could perhaps be the reason for your "skinny fat" look. If its feasible/possible try making a majority of your own food at home, or college, or wherever; eat more healthy complex carbohydrates and complete proteins (don't rely on protein shakes a lot of people seem to do this?), when I was at college I would buy tons of sweet potatoes and clean them, and heat'em up in my dorm's microwave, in other words, preparing your own food might be a bitch at college, but its a necessity to succeed in BB.
Perhaps you could also trying switching up your "5/3/1" routine. As you probably know the body adapts rather quickly and needs to be shaken out of its complacency via the utilization of different routines and eating habits. So eat more, beneficial foods, on a consistent basis, and change your routine and you should start to see some type of results. Hope this helps a bit.....