It’s not just the people who bought in 2004-2006 who are stressed; it’s anyone who has maxed out their financing during that time frame. That’s a lot of people by now.
There will be a few people from 2002/2003 who could also lose even if they didn’t max out their financing if the prices decline below that point. And that remains a possibility. If you’re under water in terms of equity and you lose your job or get a divorce it doesn’t matter when you made your purchase.