Yes to what you said. The debt levels and higher interest rates is pretty much what I write about all the time. With that said San Diego will still be priced higher then the most everywhere else. Especially if you use a price to income measure.
Once interest rates rocket San Diego will get punished and prices will drop. However it will not be proportional and affordability will still be elusive to many people, especially those without large downpayments.
