OP should be thankful he can qualify for direct cheap fixed rste loans. Many people can’t and that’s why they are stuck. Some people can’t buy their first home because they can’t qualify and some can’t refinance because they can’t qualify. So if the OP can qualify, I’d milk it for what it’s worth while you still can.
There is always a greater fool that you think you are. Just as long as you can outlast the other fools financially.