We added a granny flat/companion unit to our house. My understanding, from that process, is that if you add bathrooms, bedrooms, etc you DO have to pay more for impact fees. For our case, a detached unit on the same lot, the permit fees included this charge. (It was not an inexpensive permit.)
Also – if you do a full tear down, you end up with higher property taxes – it’s why some “tear downs” will keep one wall standing so they can claim it’s not new construction, but a “remodel”.
If the property has any issues like being on a canyon with slopes, etc, you might be limited to the same footprint of the original house. Friends bought a burnt shell in Mission Hills – built a new house – but were limited to the same footprint. Because of the canyon and new codes about slopes, they could not get permission to get any deviation from the footprint.