Another alternative is to buy a cash flow rental that you and your wife could live in as empty nesters. The tenants pay for it over those years and by the time you move in,it will be a cheap place to live and partially paid off.Presumably it will also be cheaper than the house you are renting for an entire family, so if things were to go south with your earnings and savings you could downsize into your own home.
If it works out get a few more and then buy a nice house with payment assistance from the cash flow.