I spent my middle and high school years in a smallish town in central Texas of 20,000 inhabitants. There was a reference to “the other side of the tracks” for the poorer areas which were in the North part of town, near the more scenic hills.
As a practical matter, the railroad line did make for a louder neighborhood and could delay access to the hospital.
