You have to lower it to just above the knees otherwise you will have almost zero acceleration path. The problem might be how you are lowering the bar to the knees. Most people do it by bending the knees (going straight down). The bar will thus be in front of the shoulders and that will be a very inefficient pulling position.
What you want to do is:
1) BEFORE YOU START LOWERING THE BAR bend the knees slightly (go down about 2 inches or until the knee angle is about 150 degrees)
2) From that position lower the bar to the knees BY PUSHING THE HIPS BACK (imagine doing a romanian deadlift) the knee angle actually doesn't change throughout the lowering part. While doing that, bring the bar to above the knees by sliding the bar on your quad and keep your arms perfectly straight.
3) When you reach the knees quickly reverse direction and try to accelerate the bar as much as possible by using the hips, lower back and legs (imagine doing the reverse movement of the lowering part as fast as humanly possible). This will allow you to reach the hips (power position) with a lot of speed already and from there (without stopping of course) you explode upward. The big problem is that people will bend their arms too soon (to bring the bar to the hips) and it kills momentum. The arms should not bend until the bar has been launched.