I agree with your sentiment. People are so sedentary they have no concept of the work capacity our forebearers exhibited every day. I've talked to my Grandpa about this a lot, and it seems like back in his youth, people walked all over the place because a lot of folks didn't have cars. I know my Grandma did a lot of farmwork with a big, ridiculous Belgian horse, and it was light work considered suitable for a young girl.
This was considered normal. I often think about how now, if someone was to walk 5 miles in an otherwise sedentary day, they might consider that a huge burden, but 70 years ago it would have been considered a day of relative ease. I know around the turn of the century Bernarr MacFadden routinely walked 15-20 miles per day from his home to his office.
OP: What I'm getting at is, no, it's not necessary. The amount of rest you take should be determined by the amount of rest you need to keep your performance up, which is a function of your recovery and the severity of your training. If your training is not too demanding, you don't need much rest.