No, not using a mortgage broker and yes, I’m trying to refi with the current lender.

Try opening your field up to other lenders. I know someone who refi’d to 4.675, 30yr fixed. One thing that helps is good LTV. They were getting the run-around from banks until their original lender saw that they were looking to re-fi and actually had an ‘ok’ offer (5.1%). The original lender then came in at the 4.675. The lenders are playing games out there.