Sentiment Analysis on Enron's Emails with Apache Spark(mammothdata.com)
mammothdata.com
Sentiment Analysis on Enron's Emails with Apache Spark
http://mammothdata.com/sentiment-analysis-on-enrons-emails-with-apache-spark/
2 comments
If this type of analysis can't answer "who Ken Lay likes best", maybe that's not the best question to use as an example.
"Who did Ken Lay like best" was originally going to be the title of the post, until we realized that you can't actually answer that question. It was someone else's idea anyway, but I'm not naming names!
That said, I only had so much time to work on this, and there was a limit to how much depth I could go into. I wanted to go deeper and introduce some network science / social network analysis but had other priorities.
But still, the point was just to illustrate a cool application of that particular technology stack and pique people's interest in it. If that interest helps us land business, that's a nice outcome. If not, maybe somebody learned something or came up with an idea for their own project, as a result of reading it. Those are nice outcomes as well. :-)
There are plenty of more interesting questions ("when did Enron employees start getting nervous or alarmed?")
That is a pretty good one... maybe somebody else will take the bull by the horns and write a post on that. If not, maybe I'll do it later for a "part 2".
"Who did Ken Lay like best" was originally going to be the title of the post, until we realized that you can't actually answer that question. It was someone else's idea anyway, but I'm not naming names!
That said, I only had so much time to work on this, and there was a limit to how much depth I could go into. I wanted to go deeper and introduce some network science / social network analysis but had other priorities.
But still, the point was just to illustrate a cool application of that particular technology stack and pique people's interest in it. If that interest helps us land business, that's a nice outcome. If not, maybe somebody learned something or came up with an idea for their own project, as a result of reading it. Those are nice outcomes as well. :-)
There are plenty of more interesting questions ("when did Enron employees start getting nervous or alarmed?")
That is a pretty good one... maybe somebody else will take the bull by the horns and write a post on that. If not, maybe I'll do it later for a "part 2".
It's not that kind of an "analysis."
More like a, you know, "Look, Ma -- I got Spark, Hive, and HDFS on my resume now!" analysis.
More like a, you know, "Look, Ma -- I got Spark, Hive, and HDFS on my resume now!" analysis.
did you do this as part of a coding interview challenge ?
LOL, no. It's just a fun application of some neat technology. We like to publish those kinds of things on our company blog. It's a chance for our staff to do something fun in between engagements, it helps promote the company, and somebody may learn something from it. We see that as a win-win.
Fine, fine I get it .next time I have an interview I'll solve the silly puzzle in Scala. Got it. XD
There are plenty of more interesting questions ("when did Enron employees start getting nervous or alarmed?") that can be answered by this dataset.