In which case I would agree the trespassers are in the wrong.
This notion cannot be more incorrect. Property rights came long before the nation state. Similarly, I do not need govenment to tell me my income is mine. It is mine because I voluntarily contract with an employer who provides me a predetermined wage. Government has nothing to do with this rightful contract.
Indeed though, you are unintentionally correct when you say someone bigger can come along and take what it wants. Our government does just that and they use democracy (mob rule) as the ultimate ideal of ethics to justify it. Other governments have made similar asinine ethical arguments for the same purpose -- e.g., Lenin and Hitler, etc.
The State is an agency that exercises a monopoly of ultimate arbitration in cases of conflict involving itself; it is never in the wrong, by definition. Even more insulting is the idea that it must steal from me in order to grant me this "privilege of justice".
You use the "might makes right" argument to justify government authority. If this were the case then civil society would not exist. There would never have been a large segment of society that recognized the idea of property rights. We would be living in a complete slave state and not the USA if this were the case.
Be ready to embrace the anti-USA, bizzarro world where government is good and just and individualism is immoral...here it comes.