There is really no advantage not to refi. If you refi-ed at a lower rate, kept paying the same payment and did not take cash out, the new loan would be paid off earlier than the original would have been.
Now, I agree with Raybyrnes that when you realize the monthly savings by paying a lower monthly payment, you do extend your loan and increase the amount of interest due but you can control that variable.
