You’re right. Two friends who purchased homes in the late ’90s and checked into moving up to bigger houses, realized they are priced out. Now they wish they’d bought the $300K house in the late ’90s instead of the $180K house, which they bought and is now worth in the 700’s. At the time, they didn’t know home values would skyrocket. These same friends refinanced into 15 yr mortgages when rates went down, instead of buying up or doing the almost-mandatory kitchen remodel.
On the other hand, since the highest portion of the cost of a house, the land, is fixed, the incremental cost of square footage goes down. A lady I know just moved up from an average neighborhood to a luxury home, by just paying a couple hundred thousand dollars more. Apparently she was surprised by how much more house she could get with just a little more money. In her case, she bought her home a long time ago, and their wages had gone up enough that they could easily afford to move up.