According to the census bureau about 25.5% of San Diego households had income that exceeds 100K. Nearly 11% of households have incomes exceeding 150K.
With the median price in the 450K range or so, I believe that the number of households who can afford the median house must be somewhere between 12-15%, assuming conservative DTIs.
The pseudo stat 3% number thrown around above is off by at least a factor of 4.