To me, 4100 sf seems to be serious overkill, and far more room than most people can possibly need (unless they have 3 kids, a live-in maid and a live-in nanny, or some kind of in-laws). Though if you’re downgrading from a 7000 sf house in NC, that’s a moot point.
Poway gets really hot in summer, hot and sunny, and our electricity is among the most expensive in the country (32c/kwh above 200% of baseline, if I’m not mistaken). Again, for you that may not be much, but most people greatly underestimate the cost of cooling 4000 sf through Poway summer.