I say dump him and you will probably both be happy!
P.S.- I felt really bad about looking at 25+ houses and only making 3 offers,well it was all worth it on the 4th offer. Our realtor had a lot of patience and earned every penny in my mind. She was usually quiet when we were actually in the house and I remember thinking that was strange for a sales person,but when we asked for her opinion she would give it.