If you are looking at it from purely a test score perspective, you can’t go wrong with either. Grades schools are excellent in both.

Torrey Pines probably does a little better on the high school level than La Jolla, but man is it huge (> 2600 students). I think more of the La Jolla kids end up doing private school for high school than the Carmel Valley kids, and there actually is a not so insignificant amount of choice students in La Jolla High – so it’s less of a closed community at that level than would seem at first glance.

I think that given OP’s situation, they’d be happier in Carmel Valley, perhaps up in Torrey Hills.