This is exactly right. If you are a salaried developer or lead who has not had too many jobs in the past, your chances of 'people you know who know how good you are at your work AND have the authority to sign or highly recommend you for a contract' are pretty slim.
I am currently doing this. What I am doing on the side or the companies I am doing it for does not compete in any way with what my day job company produces.
Edit - I do have the problem you are describing though. Where to find clients who will pay a reasonable rate. Don't mean to spam but since this seems relevant, full story here - https://news.ycombinator.com/item?id=7821974
Remote consulting. I setup a C-corp. 3 Full time offshore employees. Still have my day job as a full stack enterprise developer but what I am getting to learn about running a business and helping people 'produce their best' is phenomenal.
I look at my role as a really strong conduit between client and my team. I can speak both their languages and since I am a developer myself and the team is so small, I can point to the exact changes done or required to make something happen. My current client loves this.
Biggest hurdle - finding clients who have the need and budget.
(Btw, I am a long time lurker, occasional commenter but first time Ask HN poster. Stupid question - Does anyone know how to make a post appear under 'Ask'? My post appears under 'New' but I can't see it under 'Ask')