You’re actually “wrong” in a very subtle way (this is leaving aside the obvious opportunity cost issue, which many others already pointed out). It all has to do with risk premiums.
The interesting thing is that a “buy vs. rent” analysis would – surprisingly for most on this board – conclude that the housing market is fairly priced, as you noticed.
However, the same analysis done at any point in the past decades would have shown buying to be a screaming deal when compared to renting. Simply put, buying a home has always assumed an implicit risk premium, which has now all but disappeared.
To better understand this, I recommend that you read a very, very good housing paper at:
The section “Using a zero-risk premium” precisely addresses your observation. I’ll cite from it:
“Could it be that house prices in the past 30 years that we have data for have always been ridiculously cheap and that it is only now that house prices are moving towards fair values?
Not likely. These calculations assume a zero housing risk premium. But who cares about a risk premium? After all, this is a home, not an investment.
Unfortunately, this is the equivalent of saying that if you hold stocks for a long enough time period (a few decades), and you are sufficiently diversified, then history shows that stocks have proven to be no riskier than bonds, and therefore one should apply a zero-risk premium to stocks.
On this basis, today’s Dow Jones should be north of 50,000 instead of 11,000, taking the PE ratio from 19 times to about 100 times or more. Of course, many did start to think that way in the late 1990s, with the publication of ‘Dow 36,000’, one of the more notable book titles of the tech-bubble era.”
