The taxpayers moving out of state include those who would have been in a position to buy properties. The non-taxpayers entering the population (including births) are almost irrelevant to the pricing of homes because they don’t participate in the sales market.