I hope you are not implying that poorer people do not get in trouble for sexual harassment or worse.
This recent uptick in wealthy people being brought down by it is only newsworthy because they were previously seemingly immune to the normal rules everyone else lives by.
Oddly, there are some amazing young guitarists... though obviously, anyone who is already great has likely been at it for 10+ years, so there may be a significant shortage 10 years from now.
Its fine to quit when other peoples goals involve using your man-power for their own gain and you see less and less reason to continue being a part of it.