To me it would come down to two things:
1. The distance from the foot of the slope to the house
2. Whether the house on the hill has any windows that could look into my back yard.

I’m not as picky if the uphill neighbors could be in their back yard and see my yard, but I don’t really want them in their house doing that.

Distance to house: I’d be good at 30 feet plus, if I really liked the house 25 feet may be OK in some cases, but I really wouldn’t go smaller than that. Just my preference. In 25 feet I could do a lot of underground drainage to make sure the runoff stays away from my house. I would certainly go with oversized underground drain pipes. Way oversized. Up here in my neck of the woods, there are some newer yards with a 10-15 foot deep back yard then a very steep slope. That would rule them out to me.

A third factor: who owns the slope? If it is your yard, you can ensure it is properly maintained with vegetation to keep the soil stable. If it is your uphill neighbors or the HOA, that’s a different story…you could be looking at mud coming down that hill some year.