my middle delts were lagging too, i brought them up but now they make my traps look a lot smaller ha.
but anyways what i did to make mine grow a bit was i made lateral raises priority. I would do 3 sets with dumbbells in the beggining of my workout and i would do another 3 sets seated on a machine later on in my workout close to the end.
I dont know if this helped but since they are a smaller muscle and already not used to doing a lot of weight i started with 25lb dumbbells for sets of like 15 and weekly i raised the weight and lowered the reps. Then just did them one arm at a time when i got to 40lb dumbbells and when i got to 45lbs for sets of five i dropped back to 25lb dumbbells and hit sets of 20 and did the same thing til i got back to 45lb dumbbells with one arm for sets of 10.
you could also try increasing frequency, like also hitting your middle delts between sets on another day like a chest day
on another note i wouldn't ignore isolating every muscle apart of your delts. I'm very prone to shoulder injuries and i learned the hard way that isolating them is pretty important even if you just do it as a preventative with light weight and high reps plus if you hurt yourself thats going to be one of the first things a physical therapist is probably going to tell you to start doing.