You can always call the mangaer of that agents office, explain the situation and then ask to be reassigned to another agent in the office. That way, the first agent will still probably get a referral fee if you end up buying that house. But you wont ever have to see him again. You may want to look up the office and see if there is any other agent that you might want to work with based on experience, etc. Then you can have some names to give the manager and you won’t get newbie. Then the manager will run interference and He/She will get the nasty phone call from the original agent. Not you.