We used to rent an in-law unit in the University heights area where most of the properties have that arrangement. The owner lived in the front house, although he hired a property manager to deal with maintenance. There were two seriously awkward moments; one was when he passed out drunk in his yard and we had to nudge him to make sure he was alive and breathing and ok. The other was when a guy I can only assume was some sort of collections agent knocked on our door and asked questions about the property owner, mostly when he was normally home.
Honestly though, I agree with Rich; having had some bad neighbors, the ability to pick (and oust!) your neighbor would be a huge plus. Just try not to pass out drunk in your yard!