I was in a similar position 3 years ago. I jumped into an open university maths degree at the second year, have done half of the final year and have moved onto a masters degree. All part time, for me this was the best way, as another commentator has said, it is easy to get lost among the vastness of the topic - I am still pulling together the threads of how all these things fit together. I would recommend the journey for the reasons you list, also it has proven useful in pursuing my interest in machine learning and general artificial intelligence. Take the plunge, you will probably not be disappointed.