It’s actually increasingly difficult to build in any areas of a canyon now. Some cities have ridgeline ordinances that protect the top of canyons and hills from being built. Canyons would typically also have way too many biological impacts and hillside ordinances, making them virtually unbuildable. Most of the remaining areas to build in a city like Carslbad are disturbed or been used for agriculture. These carry less impacts and make it more feasible to build.
We’re actually lucky (or unlucky for some people I guess) that a good portion of the County was built long before the strict land planning policies of today. Many of those beautiful canyon subdivisions and hilltop/bluff homes are no longer feasible.