You would be upside down.. if you can afford the payments you will be fine. If you want to sell though that is when you will run into problems.
I don’t know what the refinance would be like also at the end of the 15 years. If you had enough equity by then that the loan terms made sense you would be able to refi, if not, you could be in trouble. I’m sure more can elaborate further.