Interest rates are low. Why not refi into a 30 year fixed which lowers the monthly payment and allows for more manageability. Additionally you retain the cost basis for tx purposes. No way are you going to be comfortable going from a Million dollar place to a 500K place unless your kids have moved out of the house. The fixation on having the house paid before the kids start school is not necessarily a bad thing but if it is impacting your health then refi.
If you are going to sell he place then I wouldn’t look to buy any time soon. May simply want to rent something comaprble until market finishes correcting and look to buy back in at that point in time.