It's no problem at all. I'm constantly interrupted during my workouts. I don't think it's hurt me one iota. If I'm twenty minutes into the session I will sometimes stop it and then do another twenty minute session later on in the day. No harm done.
If I'm only a few minutes into the session I will go back to it and sometimes repeat what I have already done, or restart where I left off. It all depends on how long I was away from the workout.
These are the advantages (and disadvantages) of training at your own home Gym.