Less lower back work, and more hip and glute work.
Stretch calves, hip flexors and quads. With no weight, drop into a deep squat position. Use your elbows to push your knees outwards in the bottom postion. Keep your butt low, and your knees out.
Strengthen your hips. The best way I have found is to; Wear my lifting belt. Wrap a resistance band over the belt, then stick my feet through the loops of the band and shuffle around. Basically, you have to use the glutes and hips to resist the band, which is trying to fold you over. This will teach you to stay upright, and not lean over.
Do front squats for awhile. Having the bar in front of your body, and in a "higher" position will keep you more upright. As you front squat, look at your waist, or belt line, or zipper. Make sure upright. As long drop into the hole, continue to keep your hips under you and your belt line/zipper/waist line visible. This will keep your spine neutral, and stop you from going into over extension in the lower back.
After front squats, try some light Back Squats, with a pause, for low reps. Back squat, then pause in the bottom position. While down there, force your knees out, drive your head and shoulders back and pull your hips forward and underneath you. Focus on "feeling it" in your glutes and the outside of your thighs.
Do some lunges, one leg at a time.