We had a similar plan to yours when our kids left, we were going to downsize so we sold. But we ended up buying a bigger house! And we are glad we did. We get visitors often and all the space is handy then.
We have even had kids and grandkids move in temporarily, so all space is used then also. I know some people go smaller just so kids cant come home, but we enjoy being the hub of the family. Our house will be remembered as where all the great memories happened. It makes us happy.