You are not screwed, you’ll survive and will be able to make your monthly payments. But I think your decision is not good, financially speaking. If being the “owner” is so important to your psychological well being, then go ahead, but it’s going to cost you dearly. You’re paying almost half a million dollars for a house in an inland area, a house that is probably not worth more than $250K.
We know that RE cycles in So Cal are very long, of 10-20 years duration. (Problem is, many people have only 3 year memories!) Prices just peaked in 2005 and we have several years of home depreciation ahead of us (as it happened in 1990-1996, 1982-1986, and 1973-1976).
How did I arrive to the $250K valuation above? I guess you could rent an almost identical house for $2K a month (gardener included), without the headaches and added expense of ownership. You think your baby will mind whether you own or rent the exact same house? I don’t think so. In fact, you’ll spend more quality time with your family if you don’t have all those trips to Home Depot. And you’ll save a fortune too.
Unfortunately for current buyers, in 5 years, this house will be worth at most $485K, possibly a lot less, and once you factor in inflation, you’ll realize you lost much more than just those $3500 a month payments. If I were you, I would get out of the deal, forgo my deposit and rent a house on the same block if I really liked the neighborhood.