You mention they'll gain technical skills but they'll also learn the soft skills. This is often forgotten and not learned in school. They'll become better communicators with email, in person, and over the phone. They'll gain experience with deadlines, costs, prioritization, handling clients, and many more soft skills.
These soft skills will put them leaps and bounds ahead of other fresh graduates at their first job.
I recently launched a startup that came out of a hackathon. It was and idea I had for for several months and used the hackathon to get it started. The team meshed well over the weekend but I could see the holes if we stuck together long term. Also, they wanted to do it as a side project whereas I was going to do it full-time.
My advise would be to let everyone think about what they want to do with it and then have a meeting a week or two after the hackathon. Everyone needs to be honest on how much they can contribute, what skills they bring to the team, and how serious/passionate they are about the idea. You'll probably find out that most were really excited that weekend and couple days following but then the excitement begins to fade away.