Because your ass isn't the pivot point - your lower back is. To correctly move at that pivot, your upper body needs to make a "C" shape. People with weak abs make a "V" shape and can't get themselves curled off the ground.
If it helps, think about tucking your chin into your sternum, and then your upper chest into your belly button. That will help you get there faster than stuggling and kicking your chicken legs off the ground