if you’re planning to stick around in your , I’d refi into a 30year fixed while you still qualify imho. Who knows what happens financially to you in the future. You might not be able to qualify moving forward.
Don’t feel too bad about buying in 2004. We did also. But frankly I don’t care if things on my primary depreciate, because I wasn’t planning to make money off of my primary to begin with.