My feeling is it is reasonable for you to want your own representation for the transaction. The seller’s agent who showed you the property will still get paid, so it’s not like you had him show you around town only to have you switch to another agent at the last minute.
Really, there’s no downside to finding your own agent. Be upfront with the new agent that you have been shown homes by the seller’s agent, but my guess is they aren’t going to turn you away.