Assuming no kids (schools), living in a “poor” area isn’t necessarily much worse. Most crime isn’t random. As long as you don’t involve yourself in things that you shouldn’t be involved in anyway (hard drugs, gangs, hookers), the likelihood of being messed with is low.
I knew a nice lily-white Jewish girl who lived in Spanish Harlem in the early 2000s — she said the only time she was robbed was after visiting a friend in a nice part of the West Side.