Since the value of your property is solely based on the government restrictions, then it follows that the government can tell you what you can do to your property.
We don’t let people open machine shops in residential neighborhoods because that would be a nuisance and diminish the value of everyones property next to it.
Since land is a scarce commodity unless you know how create more of it, then it should be regulated very highly and should not be viewed as any other productive item like machines or the internet or even the stock market.