Another problem is that useful effects that might potentially exist could not be expected to be detected in many studies that are done, though differing studies likely could detect them.
Not that I’m a fan of HMB (as personal opinion I think useful effect, if any, must be quite mild) but back in the early '90s when Bill Phillips was really touting HMB and getting completely slammed in Internet discussion on the matter, I put up a post, which I didn’t expect to get anywhere though it was a good idea, suggesting a means of resolving the question.
Phillips could have provided HMB and placebo bottles to the UF Dept of Exercise Science, and I had a professor in mind who I was pretty sure would have liked the idea, and certainly would have been able to do it, of running a study where several hundred weight training subjects drawn from the student population would be put in matched groups and allowed to train and eat ad libitum. (You are not going to be able to either obtain this many subjects if you demand you control their diet and training over an extended time, plus the costs of managing that would be huge.) Duration of the study: say six months. Body comps say pre, mid-study, and post.
If there were an average effect to use of HMB such as improving LBM by say 3 or 4 lb in 6 months, which for all I know it may do, I expect this sort of study would be able to establish it to reasonable significance, thanks to the large number of subjects.
If however that is all that HMB does, I’d completely expect an 8 week study with 10, 16 or whatever subjects per group to fail to resolve such a small effect.
Of course it wasn’t done, and to this day I expect we still don’t know for a reasonable fact whether HMB may have a small useful effect such as that, or not.
In the case of the herbs, I expect that many useful effects if of the magnitude that might reasonably be expected would be indetectable by the methods used.