If I were to downsize, I could probably find 3/3 condo that would cost almost the same as how much I originally paid for my SFH, if I were to factor on the additional $200-300/month extra I would need to pay for HOA on the condo.

So there is no point in me downsizing. Folks that bought much earlier and already paid off their home, it would make even less sense probably.