Yeah we talked about that as being the obvious next step but we had neither the time nor the skillset off the bat for it. A noSql DB would probably be the better option for that as well. (correct me if i'm wrong)
This is the result of an 8 day final project for DBC Chicago. Our team scraped over 140,000 headlines of several news agencies stretching back several years. We then took those headlines and fed them through the AlchemyAPI sentiment analysis engine to assign each one a score. They were then plotted in a couple different ways using D3.
This is far from perfect and even farther from scientific. It was done in 8 days by some passionate amateur developers. It was however a lot of fun and very interesting.
You can read about it and the team in more detail on the repo page here:
It's html, a lot of fancy CSS transforms, and jquery/js.
The headlines are currently stored in just a static json file with urls I picked out.
I found it surprising that it's actually easier to get quality "real" headlines (fark, onionlike.tumblr.com/) than believable onion articles.
The next challenge is having the headlines stay up to date.
We're currently planning on seeding the database using the onion rss feed. The problem is filtering out the headlines which are too obvious.
Any ideas?