We moved here for the same reason (SoCal beach lifestyle and ended up in South Carlsbad (Encinitas school district). You’re right about PB and OB which are dumps full of trashy drunks looking for meth, a fight, or both. Not an area I’d take my dog much less my kids. Last time I went to Dog Beach I got sick for a week because of who knows what was in that water. Del Mar has a much nicer dog beach.
I rarely venture further south than Del Mar. La Jolla is way too crowded. I can get to Del Mar in 20 minutes via Coast Highway and I’ll pass 20 restaraunts that are good to excellent. Shopping is right around the corner, the beach is a few minutes away, and the schools are great. Not much to complain about. The only change I’d request is that the Pizza Port Brewery people would add a 3rd location (other two are Solana Beach and Carlsbad Village) smack dab in the middle of those two which by my account would be where the Yellow Coyote in the Forum currently resides.