There is no real cause for concern. Tamoxifen is prescribed by physicians even as a prophylactic (preventative agent, or more precisely an agent to reduce risk) without them fearing liver problems, when there is no liver problem in the first place (it can be different if there is.) Clomiphene isn't ordinarily used long-term for women, not for reasons of fear of hepatoxicity but because the ordinary use is one to a few treatments for fertility. It has been studied for ongoing use in men for a period of one year with no liver problems.
Bodybuilders don't report liver problems from it.
In women the combination of clomiphene (which can act as an estrogen) with HCG (which increases estrogen for them) has either rarely caused hepatic hemangiomas or has worsened existing ones. Not sure which. These are known to be caused and aggravated by estrogen.
The term "hepatoxicity" does not refer to likelihood of causing liver problems, but whether it ever happens at all.
In practice, bb'ing use shows no reason for concern. Clomiphene and tamoxifen have been safely used for a long time by very many.