If you’re still in the contingency period, it would probably be wise to back out. Don’t be afraid to use legal counsel if he/she doesn’t cooperate.
If they ask why, tell them it’s because you don’t trust them for the reasons you gave above. At this point, if you really want to buy the house, you might want to hire an attorney to work with you on the rest of the deal.
Did they recommend the title/escrow company, inspector, etc.? They might be skimming more off this deal than you think.
If you feel like they have you over a barrel, it might be a good idea to have a friend with a strong personality and lots of business sense to hold your hand through this and/or speak on your behalf — especially if you can’t afford an attorney.
