First off I would advise you to talk to your lender in both cases to discuss the situation. This is by far the most important thing to do.
Second, there will be good advice on this board, but take it with a grain of salt and seek qualified professional guidance on the situation.
Third, yes any loans made to you that have not been used for original purchase money for a home are loans that the lender can come to try to get back from you. So even though the refi you did was for another home, technically (I believe) that they can come to try to get it. Again, this is just an opinion so don’t take it as anything more then that.
Fourth, repeat of first, TALK TO YOUR LENDERS! Best of luck.
