Analyze Your HN Posts with Watson User Modeling(hn.mybluemix.net)
hn.mybluemix.net
Analyze Your HN Posts with Watson User Modeling
http://hn.mybluemix.net/
25 comments
Seems to be down for me: 404 Not Found: Requested route ('hn.mybluemix.net') does not exist.
it is going up and down, it seems. I am getting results though
To clarify, hn.mybluemix.net is not an IBM app. A HN user built that app and it appears to crash often
Did this with Twitter:
twurl "/1.1/statuses/user_timeline.json?count=200" | jq -r ".[] | .text" | pbcopy
paste it into http://watson-um-demo.mybluemix.net/demo
87% Openness, 5% Agreeableness, that's funny.
paste it into http://watson-um-demo.mybluemix.net/demo
87% Openness, 5% Agreeableness, that's funny.
This inspired me to do the same with reddit, so I threw together this function in zsh (should work with bash too):
Never user/heard of jq before, it's a pretty nice tool.
function redcom(){ ! [ -z "$2" ] && i="&after=$2" || i=""; data=$(curl -s "https://www.reddit.com/user/${1}/comments.json?count=100${i}"); j=$(jq -e -r '.data["after"]' <<<$data); echo $j; (jq -e -r '.data["children"][]["data"]["body"]' <<<$data)>>${1}-redcom.txt; echo $(echo -e $data|wc -l) lines; ( [ "$j" = "null" ] || [ -z "$j" ] ) || redcom $1 $j ;} ;
Never user/heard of jq before, it's a pretty nice tool.
I got 92% Openness, 10% Agreeableness on tim333, HN
Maybe they all come out a bit like that?
Maybe they all come out a bit like that?
Is there any insight as to how this works? I got an Openness rating of 97% and a Harmony rating of 100%, both of which I know are not true. (I also received a Love rating of 1% under my Needs, although that's pretty accurate.)
I, too, got a "Love" rating in the low single digits. That's probably not too far from the truth in a global sense, but remember that in this case, the API is working with a local context (HN comments). I'm probably unlikely to have discussed the subject of love on HN, or anything tangentially related to it, or anything that would somehow give an indication of my need for love. Now, if you were to run my Facebook history through the same process, I imagine you'd find a slightly different analysis.
To some extent, our personalities are our personalities. We behave, on some level, the same in every context and in every community. But the extent to which that's the case is up for debate. We probably use different approaches, or if you prefer, we show different aspects of our personalities, in different contexts and in front of different groups. That's why it's extremely difficult to take one context (HN, for instance), and extrapolate universal characteristics from it.
Even within the set of HN history, I got some oddball results. For instance, Watson considers me very "Fiery" (51%) here. There are plenty of areas in my life in which the word "Fiery" makes a bit of sense. HN isn't one of them.
To some extent, our personalities are our personalities. We behave, on some level, the same in every context and in every community. But the extent to which that's the case is up for debate. We probably use different approaches, or if you prefer, we show different aspects of our personalities, in different contexts and in front of different groups. That's why it's extremely difficult to take one context (HN, for instance), and extrapolate universal characteristics from it.
Even within the set of HN history, I got some oddball results. For instance, Watson considers me very "Fiery" (51%) here. There are plenty of areas in my life in which the word "Fiery" makes a bit of sense. HN isn't one of them.
A problem I've observed with these kind of blackbox systems is that the process from input to output really is a mystery.
When the results are right, they're just "right" so you should accept them, when they're wrong they're actually also right by whatever magical hamster wheel is operating inside of the thing and you just don't "get it".
The problem is that humans like to have some clue as to how the results were derived, something easy to explain that gets the gist across. Something like "Watson counted all the words you use and compared them to different reference lexicons to arrive at the score". This provides a little bit of context so we understand the semantics of the result and how to consider them and reason with them.
But for all we know the results we're seeing are from some arbitrary stochastic method:
openness=rand(90,99) harmony=rand(90,100)
etc.
For things like this to be accepted by the users (humans) there needs to be a quick explanation for how this works otherwise we get head scratchers.
When the results are right, they're just "right" so you should accept them, when they're wrong they're actually also right by whatever magical hamster wheel is operating inside of the thing and you just don't "get it".
The problem is that humans like to have some clue as to how the results were derived, something easy to explain that gets the gist across. Something like "Watson counted all the words you use and compared them to different reference lexicons to arrive at the score". This provides a little bit of context so we understand the semantics of the result and how to consider them and reason with them.
But for all we know the results we're seeing are from some arbitrary stochastic method:
openness=rand(90,99) harmony=rand(90,100)
etc.
For things like this to be accepted by the users (humans) there needs to be a quick explanation for how this works otherwise we get head scratchers.
Please see my other 2 responses in this thread for some insight. I think I posted them about the same time you posted this.
We plan to add more information to our docs soon about the service, including a description of each of the traits, and possibly reference some of the many data sources used.
Meanwhile, you can do a search on "IBM System U" (the project's not-so-internal code name.) This particular slideshare.net prez has some great info on the methodology, validation tests and references: http://slidesha.re/1ri0vPV
Meanwhile, you can do a search on "IBM System U" (the project's not-so-internal code name.) This particular slideshare.net prez has some great info on the methodology, validation tests and references: http://slidesha.re/1ri0vPV
It's using this Watson API: http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercl...
and using the sample Node project to show the results. I am still exploring it myself :p
and using the sample Node project to show the results. I am still exploring it myself :p
I also get rather high Openess and harmony scores, though I'm particularly amused that Hedonism is scored. How can I be 72% sympathetic and 47% coopoerative but not agreeable?
The IBM documentation doesn't really say anything about how these numbers are calculated
The IBM documentation doesn't really say anything about how these numbers are calculated
I don't even have scores for stability and practicality. Hmmm.
This is very cool. Although I don't know how accurate this is, if calibrated and tuned to yield a certain degree of accuracy it will have a variety of use cases.
For example -- When interviewing someone, being able to run their github username (if known of course) to analyze their commit messages, comments, discussions. Or even their hn, reddit, twitter user names (if the usernames are linked with their first names, nothing creepy). It will potentially help to identify candidates that are downright rude, arrogant etc.
Or analyze internal mailing lists, hipchat/slack channels for co workers who are potentially burnt out.
For example -- When interviewing someone, being able to run their github username (if known of course) to analyze their commit messages, comments, discussions. Or even their hn, reddit, twitter user names (if the usernames are linked with their first names, nothing creepy). It will potentially help to identify candidates that are downright rude, arrogant etc.
Or analyze internal mailing lists, hipchat/slack channels for co workers who are potentially burnt out.
>>It will potentially help to identify candidates that are downright rude, arrogant etc.
This sounds very dangerous to me. I assume when recruiters and/or lead engineers decide to reach out to me via LinkedIn, they did their homework on me. I purposely link to enough stuff for them to realize "smtddr" is my handle. The same blog & youtube channel in my HN profile is also in my LinkedIn. But I expect a human to look, not some computer judging me. I can totally see people getting lazy and just doing stuff like only filtering for people who rate 80% on openness or something. Then everyone will start grooming their posts simply to get positive results... then someone will create a social website that claims to block those scanners so people can say whatever they want.
It just forces people underground and the filters won't work anymore since at that point you might as well assume everyone is gaming the system.
(fwiw, I'm also against standardized tests. Anything that forces a whole group of people to start grooming themselves for a very specific measurement kills diversity, imho. Since the very term "standardized" kinda goes against the concept of diverse... and people become lazy and just rely on such tests to make or break the deal)
This sounds very dangerous to me. I assume when recruiters and/or lead engineers decide to reach out to me via LinkedIn, they did their homework on me. I purposely link to enough stuff for them to realize "smtddr" is my handle. The same blog & youtube channel in my HN profile is also in my LinkedIn. But I expect a human to look, not some computer judging me. I can totally see people getting lazy and just doing stuff like only filtering for people who rate 80% on openness or something. Then everyone will start grooming their posts simply to get positive results... then someone will create a social website that claims to block those scanners so people can say whatever they want.
It just forces people underground and the filters won't work anymore since at that point you might as well assume everyone is gaming the system.
(fwiw, I'm also against standardized tests. Anything that forces a whole group of people to start grooming themselves for a very specific measurement kills diversity, imho. Since the very term "standardized" kinda goes against the concept of diverse... and people become lazy and just rely on such tests to make or break the deal)
You can test the accuracy yourself: http://www.outofservice.com/bigfive/ . It's a good idea to take that test before looking at OP's results.
But can you really judge a person based upon commits them might make?
You can read the words they wrote but you cannot perceive the tone they wrote it in....I dont think this is helpful and may dismiss candidates that are otherwise perfect except when Watson analyzes them.
You can read the words they wrote but you cannot perceive the tone they wrote it in....I dont think this is helpful and may dismiss candidates that are otherwise perfect except when Watson analyzes them.
Would love to see something like this for reddit, where I'm a more active poster on a wider variety of issues.
Another commenter mentioned http://watson-um-demo.mybluemix.net/demo above, so I threw this together in bash for reddit comments. Bit more of a pain that entering a username, but if you take this:
function redcom(){ ! [ -z "$2" ] && i="&after=$2" || i=""; data=$(curl -s "https://www.reddit.com/user/${1}/comments.json?count=100${i}"); j=$(jq -e -r '.data["after"]' <<<$data); echo $j; (jq -e -r '.data["children"][]["data"]["body"]' <<<$data)>>${1}-redcom.txt; echo $(echo -e $data|wc -l) lines; ( [ "$j" = "null" ] || [ -z "$j" ] ) || redcom $1 $j ;} ;
something like this?
http://www.redditinvestigator.com/
http://www.redditinvestigator.com/
Interesting! I like the fun guessed data. I am from Canada, game of choice is ping pong, nor do I have children.
"""
Probably from: Canada
Support OWS: Probably no or doesn't care.
Children: I do not think so...
Gamer: Only pong probably...
Like trees: Must be in a really good mood.
Behavior: Candidate as replace for Good Guy Greg
"""
"""
Probably from: Canada
Support OWS: Probably no or doesn't care.
Children: I do not think so...
Gamer: Only pong probably...
Like trees: Must be in a really good mood.
Behavior: Candidate as replace for Good Guy Greg
"""
For those that like this. There is a similar thing I saw awhile on here (I am guessing) that analyzes facebook posts and compares you:
http://labs.five.com/
http://labs.five.com/
Gregariousness
8%
Given my name I found that funny Also I know it's wildly inaccurate on several characteristics, but maybe my persona here is like that! Interesting.
Given my name I found that funny Also I know it's wildly inaccurate on several characteristics, but maybe my persona here is like that! Interesting.
Yeah, I think it's worth keeping in mind it's scoring you based on your comments on a particular kind of site which tends towards certain kinds of interactions.
Does Watson internally use observations drawn from this study[1]?
And another question---is this applicable to non-native English speakers? Do they acquire the same language habits as if English was their mother tongue?
[1]: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3783449/
And another question---is this applicable to non-native English speakers? Do they acquire the same language habits as if English was their mother tongue?
[1]: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3783449/
Now I'm consistently getting this:
Edit: In fact, entering someone else's username worked, and another, but mine continues to fail. Is it just me?
502 Bad Gateway: Registered endpoint
failed to handle the
request.
Edit: In fact, entering someone else's username worked, and another, but mine continues to fail. Is it just me?
I believe you are talking about the hn.mybluemix.net app. That is not an IBM app - a HN user put it together. It appears to be crash often
Well, yes, because that's the link in this submission.
If you picked random English words and put random numbers next to them, I wouldn't be able to tell.
But that's how the stock market works...
Not a lot of documentation on the IBM page about what the characteristics mean or how they learn a mapping from text to these categories.
Curious to know what the average hacker news scores look like? I'm imagining it is a pretty small segment of "normal" society.
Curious to know what the average hacker news scores look like? I'm imagining it is a pretty small segment of "normal" society.
See my response to minimaxir above about additional documentation coming soon, and a link with good background info on the technology. Meanwhile, here are brief descriptions of the Big 5 Personality traits:
Big 5 Personality:
- Openness - associated with curiosity,
intellect, and an appreciation for art and adventure
- Conscientiousness - associated with organization and
industriousness
- Extraversion - associated with positive and outgoing
attitudes toward other people
- Agreeableness - associated with compassion and
cooperation toward other people
- Emotional Range - associated with a sensitivity to negative emotions
For more information on systematic associations between personality and individual differences in word use, please refer to studies like Tal Yarkoni, "Personality in 100,000 words: A Large scale analysis of personality and word use among bloggers", 2010
For more information on systematic associations between personality and individual differences in word use, please refer to studies like Tal Yarkoni, "Personality in 100,000 words: A Large scale analysis of personality and word use among bloggers", 2010
Interesting, I analyzed myself and patio11.
any insight on the tech stack and how it was implemented?
any insight on the tech stack and how it was implemented?
It would be interesting to see the code.
I'll throw it on Github right away. I basically just used IBMs Node sample code which had it all there I just hooked it up to the HN api :p. https://www.ng.bluemix.net/docs/#starters/nodejs/index.html#...
100% Challenge?
Not sure what that means... but maybe this post is contributing to it?
Not sure what that means... but maybe this post is contributing to it?
Conscientiousness 23%
Ouch! How did it know?
Some are completely off, with plenty of 1% cop-outs but a few — spot on. What's the methodology? What does the need for practicality mean for example?
Ouch! How did it know?
Some are completely off, with plenty of 1% cop-outs but a few — spot on. What's the methodology? What does the need for practicality mean for example?
[deleted]
I would be curious to see if some of these characteristics are correlated with higher average karma/total karma.
[deleted]
Hmm.
404 Not Found:
Requested route ('hn.mybluemix.net')
does not exist.Shall we rename slashdot effect?
[deleted]
[deleted]
I wonder how well it detects sarcasm?
I analyzed my HN posts with the Watson API on an Oculus Rift and the singularity happened.
When the service first came up on the front page, I punched in some personal writings from my journal into it. Comparing my results from that to this HN assessment is like night and day. The HN assessment shows me as highly intellectual, imaginative, and adventurous, with a strong value for self-enhancement. The personal stuff showed me as being much more emotional, with a huge emotional range, because—surprise, surprise—I tend to write more about emotional parts of my life in my journal than on HN comments.
While this service might be useful to get a broad sense of people for marketing purposes, using it on an individual basis is like talking to a fortune teller—it could tell you nearly anything about yourself, and you'd be able to come up with an explanation to justify it.