There’s always a lot of work done without permit. Technically, you’re supposed to get a permit to add a light switch or to relocate a door.
There’s nothing wrong with “unpermited” work so long as the work was performed to safety code.
Setback limitations is one reason people don’t get permits. For example a patio cover may not be allowed under setback limitations so people would just build one without informing the city. Since nobody complains it’s never a problem.
BTW, Del Mar is well known for unpermitted work. Del Mar has strict livable square-feet per acre limitations so owners might get a permit for space with 2-story-high ceilings then come back later to add 2nd floor space inside. It’s done all the time.
“Unpermited” space is fine if it’s built with safety in mind.