Yes, you can let housing become overvalued by ten trillion dollars, and lend against the overvaluation by another few trillion, and the problem is the “infection” of letting the values come back down to earth.
This is what down payments were originally for.. the event of a decline in the value of the property. This is why the traditional Fannie/Freddie product was a 80%LTV maximum (20% down, 80% financed). The down prevents a ‘cascade’ of defaults because the value of the house exceeds the outstanding loan even if house prices drop a bit.