Good post by Batman. I mostly agree.
I think it also depends on the individual. For people with less dense bone structures (children, most women, smaller men, and people who have not put in years of bone conditioning to make up for a lack of “natural born gifts”) I actually think open hand strikes are tactically advantageous regardless of the target. There are some closed fist strikes (like hammer fists) that are more durable than others as well; so it’s not just about “open hand vs closed fist.”
For people with either naturally very dense bone structures or who are willing to put in years of bone conditioning to essentially make their fists into “rocks” punching with a closed fist focuses the forces into a smaller surface area, meaning more acute damage (chances of cutting, bruising/local tissue trauma, etc…) and of course less dissipation of force at impact before the bone structure makes contact with the target. As Batman also mentioned, balling up the fists is a natural response to stress and/or aggression.
One other “advantage” to open hand strikes is the possibility (or intentional use) of getting a/multiple finger(s) into the opponent’s eyes. You see this accidentally happen in MMA all the time due to the open fingered gloves. It’s obviously an illegal and usually unintentional tactic in a sporting event like MMA, but a great tactic for self defense since the eyes cannot be conditioned to take strikes and are a very sensitive and vulnerable target.
One possible disadvantage of open hand strikes is the possibility of small joint locks. This wouldn’t be a weakness that many people would be able to exploit of course, but if you happened to run into somebody who could, you’d be wise to prepare for it or else you might find yourself without properly functioning hands in very short order.