A contact lens that corrects for astigmatism is required to sit on the eye and not rotate around.
Astigmatism occurs when the cornea is not a perfectly round shape like a soccer ball, but distorted so its more like a NFL ball. Hence your astigmatism is at a particular axis(degree), and hence your contact lens must sit at a particular position and not rotate in order to correct for your astigmatism.
Your contact lens is designed to sit on the eye via having "weights" at the bottom of the lens (or by a system known as "prism ballast"), so the that the bottom of the lens always remains at position "6 o'clock". When training you will often scrunch up your eyes upsetting this weighted system causing the lens to roatate, and hence not correcting your astigmatism at the required axis.
This can be expected to occur occasionally, but if it happens often then the lens is considered to be not fitting properly. You may want to consult your primary eye care practioner and discuss this with him/her. There are many different types of lenses that use different materials, and are designed different in size, shape, weighted systems etc, that may suit you better, and hence give you clearer and more stable vision.
Do you know what power and degree of astigmatism that you have?