There’s a legal term called “ostensible agency.” In short, it means if someone appears to be your agent ie. you called him ‘my agent’, he in fact is your agent even if there is no written agreement. On the other hand, ‘agency’ and ‘paying a commission’ are two different things.
In my opinion (and I’m not a lawyer), you don’t owe this guy anything. You have no written compensation agreement. If he tries to collect a fee/commission, don’t pay it. If he continues to pressure you, get a lawyer.