Land isn’t worth much of anything to anyone until someone is ready to build. That’s why not much gets built during downturns – no profit.
The site value at peak was probably in excess of $300k, maybe more. Not counting fees and permits, the costs to build are about $100/SqFt, and that includes a small profit margin for the builder. Those fees do add up though, which is why in California we probably won’t be getting down to $100/SqFt construction costs.
There’s no reason that site can’t devalue back to $50,000 or even less. As far as the value of an existing home never dropping below the cost of land and improvements, I direct your attention to Exhibit A; all the towns across the US where you can still buy an average condition home for less than $50,000. It’s all relative to local wages and employment, and if this place ever gets that bad for wages and employment the prices will reflect that reality.
“It hasn’t happened yet” is not the same thing as “It can’t happen.”