Yeah, I simplified it by not using any dynamics in the generation process. I could probably add some version of dynamics using the MIDI velocity, actually, but I haven't done that yet. Also, I generated the mp3 files from MIDI using GarageBand, which doesn't help with the flat dynamics.
I trained it for about 24 hours, although there didn't seem to be that much improvement after the first 12 or so. Generating a song with the trained network actually happens almost in real time. I'm tempted to try to make it continuously generate new music and stream it, but even the small cost for the instance would start to add up, so I haven't actually tried setting that up yet.
Another interesting data point: the learned set of weights ends up being about 15MB.
I was able to find quite a bit of Bach music, actually, and I used it when I was first experimenting with this idea. At the time, I hadn't added dropout, so the output wasn't as interesting. I'll definitely try retraining the updated network with Bach.
I've definitely considered it, but so far I haven't found many single-composer datasets that are large enough to train with. I also think it would be pretty cool to try training it with music from specific musical periods.