I know you think it sucks especially since you left in great condition, but $3400 is a small price to pay to get out of a lease 5 months early and into your new home.
Not having it in writing is another reason to let it go. Maybe you can draft something up that says he is letting you out of the lease and keeping the security deposit for wear & tear. Also agree that no other compensation will be paid. That way he can never come back and say you broke the lease or try to charge you for some random damage you didn’t do.