I'm white and used to live in the Bronx around Valentine Ave. I don't think anybody I knew felt like we lived in a segregated neighborhood.
Also, plenty of people who do retail or food service work in NYC make good money, and plenty make enough to live a good life in the outer burroughs. NYC is accommodating of those that don't.
I certainly never spoke to anybody who felt like we were defacto enslaved. Just one person's experience...