It sounds like you are saying that the garage is the only thing in the setbacks? Several things come to mind. First, the authorities usually are less disturbed by a garage than by living space. Second, Maybe it replaced a previous garage and can be “grandfathered” in,or at least that could be a component of a persuasive argument for keeping it. Look for evidence of this.Does anyone in the area have an old garage in the same area of their lot?Maybe the info you get from the city will show an old garage footprint.You want to make sure it isn’t built on top of an easement, besides being in the setbacks. That could be really appurtenant(real estate humor).
If it is not on top of an easement you could probably make a case for hardship or just the fact that doing anything about it doesn’t make sense. Think greasy wheel. A lawyer might come into play somewhere again, but maybe the city would just accept it.The opinion of the owner of the adjacent property is going to possibly or likely be important as well. It might be easier to fight if you were the culpable owner. It might not be so bad anyway.
As far as traffic goes you can check for how fast business cards are piling up on the kitchen counter or other location that would be handy to a buyer’s agent.
