I think the best way to accomplish your goals is to be upfront with the agents you use from the very beginning. Most good realtors have a variety of clients that have a wide range of needs. Some of them require alot of handholding and some do not due to experience or for other reasons. I think as long as you spell out your expectations then you should be okay. Don’t sign a representation agreement so that if you want to use someone else, then you are not bound. However being courteous and letting the agent know is fair and prevents them from doing work for a client who will not be using them in the future.
Getting an unbiased opinion can be challenging. Many times you get the buyers agents who tend to be overly optimistic about any property they show you. Similarly when you go on your own and deal with the listing agent directly it can be challenging to trust what they say. Sometimes buyers bring third parties like friends with them on showings just to have another set of eyes there.
Agents that focus on giving information rather then unbiased opinions are to me, the most helpful resources for clients. Now, more often then not, a client will ask for the agents opinion on the home, neighborhood, etc and in those cases it is helpful for the agent to pass that opinion on as it was solicited. As you already may have found, some agents may push hard for various neighborhoods based on the parameters you told them are important.
As for finding it difficult to view homes you like, it is typical for people to be quite disappointed with the inventory they have been looking at, especially at San Diego prices. Try not to burn out on it and don’t let it consume you. Don’t be afraid to take a break every now and then as well.