Since I had kids I never lived in less than 1600 sf, and I think we won’t be happy in less. Could we make it work if need arises? Sure! But it would take a lot of work, time, and stress and you will really miss a lot of nice things your house had. And all the minor and not so minor inconveniences of the new place will start to really suck, and annoy various members of your family, and well, it’s just not going to be easy. Beside that, you will have to change furniture to fit, throw/donate/store a lot of other things… Can you ALL deal with all the stress? Probably your kids are going to be pretty affected by this. Not saying is not a good lesson for them, but… just hard to bear sometimes. I’d rather commute a little, than downsize quite that much. Something like 1700 sf with 4 bdr would have a much higher chance of success, IMHO. 1300 sf, 3 bdr, not much.

To me it’s not clear why you want to buy this particular house 1300 sf. Have you looked around at houses plenty? Perhaps for a similar amount you can find a house that will suit you much better.

Sure, mortgage-free is attractive A LOT but if I were you, because of the kids, I wouldn’t move. Or in the worst case, I’d rent out the big house (which can be difficult), so you can give it a try. You shouldn’t have problems to rent the smaller place if you changed your mind, or sell.