We’ve had a couple threads that tried to discuss this, but never got much interest before. I’m no expert, but a couple points.
1) The permits for doing this are often a lot more than you would think, and the permitting process, with revisions to your plans be very expensive.
2) There are lots of hidden traps to fall into.
3) Typically, you will spend a lot more time involved in doing this then most people think.
4) Lots of people do manage to overcome these problems and do rebuild houses. Some with more success and some with less. So, it can be done, just keep in mind that it will be more expensive and more time consuming and more headaches than you originally thought.
Lastly, it seems to me a hard way to get out of HOA, and Mello Roos. The main appeal is in getting to design, build and then live in your dream house. Not saving a few thousand dollars.