The guidelines (geared towards hypertrophy) seem to be:
for squatting, deadlifting, and heavy rowing: you stop when your form breaks down and you can’t hold proper positioning; the next rep would be iffy
for barbell compound movements: you stop when you reach failure; next rep wouldn’t happen unless you compromise form
for machines, isolation movements, smaller exercises: you can go beyond failure and alter your form a little bit to compensate for fatigue, or you can add partials or intensity techniques to go beyond failure (think about cheating a little bit a curl or a lateral raise or doing partials at the end of a pec deck set)
Of course this depends on your experience, strength level, goals (strength may benefit from keeping more reps in the tank), the program you’re following, and a whole host of other factors.
In the case of a 5x5 squat like you mentioned, if I was training for strength I’d probably use the first 3-4 sets to ramp up to a top set keeping one or two rep in the tank, maybe followed by a lower weight, back off set going close to failure and getting in as many good reps as possible. This, of course, unless your program specifies otherwise.