Not a knock on you personally, but this kind of thinking cracks me up. So pescatarianism is like being a vegetarian, without being a vegetarian? Gotcha. Seriously though, when I spent two months as a vegetarian and vegan, I can't tell you how many times I heard, "So you're not eating animal products, but you can still have fish, right?"
Not entirely true. He spent some time throughout the '40s and '50s eating raw beef and had other periods with no animal products at all.
Ask most people from Southeast Asia and the Mediterranean area.
As was said, if you focus on wild fish, you'll likely reduce any risks. Another good idea, since you're already limiting your protein sources, would be to only have fish as the primary protein source in one main meal per day.