Del Mar crime appears high because of the small population base. It is also seasonal. There is a racetrack there during the summer and the San Diego County Fair. You are correct there is opportunistic crime near the beach when tourists park their cars full of loot there and walk to the beach. Nearby, in Solana Beach there is a hard core element of addicts who live in an area called Eden Gardens and routinely burglarize to support their habit. Rancho Santa Fe is farther from the beach, but has its own security force which has a far better response time than the local Sheriff because they are based in RSF. I vote for RSF or that vicinity. Make sure you buy in the correct zip code or you will be paying RSF prices without the true value. You need to work with someone that knows the area and not someone who just wants to take your money. I suggest you rent out here during the season and see what goes on during the summer and how you can handle the traffic and the congestion. Del Mar was a much better place in the 70’s and 80’s, less trend and more hamlet.
P.S. Del Mar City officials are hard to deal with if you wanted to buy an older house and upgrade it. That is why the people who live there are called Delmartians…