Experiment: No Comment Scores
313 comments
Wow, this interferes with key elements of my system for voting/commenting on HN. Here are some things that don't work now:
* If a comment is sitting at 1 or 0, I'll try to avoid downvoting it unless I really think the comment subtracts value from the site.
* If a child is attracting more votes than the parent, and I think this is because the child commenter didn't comprehend what the parent was saying, I vote to level out the comments.
* If I see a comment sitting in negative territory that I feel should only be a 0 or 1 because it wasn't that bad, I'll upvote it even if I otherwise wouldn't have.
* If a thread is already dominated by a couple of high-scoring comments, relative to its age, I'm less likely to post a new top level comment. My feeling is that if a good comment just sits at the bottom because it arrived slightly too late, I've only added noise to the thread. The lack of visible points makes it much harder to gauge how a new comment will 'compete' on the thread.
* If the scores indicate that something is being widely misunderstood, I might comment where I otherwise would not have to try to restore sanity.
Of course, if this change were to stick, it would alter the character of the site, and we would all develop new voting and commenting systems. I can already tell that this change magnifies my tendency to vote with an eye toward reordering comments rather than voting on each comment in relative isolation.
* If a comment is sitting at 1 or 0, I'll try to avoid downvoting it unless I really think the comment subtracts value from the site.
* If a child is attracting more votes than the parent, and I think this is because the child commenter didn't comprehend what the parent was saying, I vote to level out the comments.
* If I see a comment sitting in negative territory that I feel should only be a 0 or 1 because it wasn't that bad, I'll upvote it even if I otherwise wouldn't have.
* If a thread is already dominated by a couple of high-scoring comments, relative to its age, I'm less likely to post a new top level comment. My feeling is that if a good comment just sits at the bottom because it arrived slightly too late, I've only added noise to the thread. The lack of visible points makes it much harder to gauge how a new comment will 'compete' on the thread.
* If the scores indicate that something is being widely misunderstood, I might comment where I otherwise would not have to try to restore sanity.
Of course, if this change were to stick, it would alter the character of the site, and we would all develop new voting and commenting systems. I can already tell that this change magnifies my tendency to vote with an eye toward reordering comments rather than voting on each comment in relative isolation.
One Idea I've had for fixing this problem is to weight the value of a click based on position on the page. The further down on the page, the more weight its gets. Therefore the most popular topic at the top only get marginally incremented as people read it first and decide its worthy.
On the same note, if an article is waaaay down at the bottom, and someone's taken the time to read it and likes it, then it should get a good solid bump. This will create a "bubble effect" and allow those treasures that are buried deep down in the comment land, to rise to the top and even accelerate as more users find out and rank them.
This essentially is a formula to equalize the comments and give them a fair playing field as they battle for the top most position.
On the same note, if an article is waaaay down at the bottom, and someone's taken the time to read it and likes it, then it should get a good solid bump. This will create a "bubble effect" and allow those treasures that are buried deep down in the comment land, to rise to the top and even accelerate as more users find out and rank them.
This essentially is a formula to equalize the comments and give them a fair playing field as they battle for the top most position.
Any chance we can see scores when we're looking at our own comments section?
Often when I notice a bump in my karma I look at my comment threads to see what was so popular.
Often when I notice a bump in my karma I look at my comment threads to see what was so popular.
But now I cant quickly skim off information from a thread. There are 87 comments in this thread at this moment. I am not interested in reading 87 comments about this experiment. I am however very much interested in seeing what the top few most insightful comments in this thread are. That feedback is very important. Right now I am feeling blind as a bat...
This change might improve voting, but it decreases the value of that voting.
I'll stipulate that the voting might be more "fair", if that is a goal. I'm not sure it should be. It might make the "game" of getting high karma more fair, but I'm not bothering with or bothered by that game anyway.
On the other hand, I can no longer use the results of this voting to help me skim a conversation. I want to know what everyone else found most valuable, without having to read through every comment on my own. I can't do that anymore. This completely breaks how I use HN.
I'll stipulate that the voting might be more "fair", if that is a goal. I'm not sure it should be. It might make the "game" of getting high karma more fair, but I'm not bothering with or bothered by that game anyway.
On the other hand, I can no longer use the results of this voting to help me skim a conversation. I want to know what everyone else found most valuable, without having to read through every comment on my own. I can't do that anymore. This completely breaks how I use HN.
PG: Will you be applying some metric to determine if this improves site feel? If so, what metric?
But don't votes also give feedback to the commentor? I have used this to learn what works within this community and what doesn't. Now, I only know by approximation to my total karma, or if i am bad, that I turn gray.
I didn't realize how much I was affected by the comment ratings until they went away. Commencing cognitive restructuring.
I can still see the score on the "edit" page, which means I can still edit my post to whine about being downmodded. Since nobody knows the degree that I am being downmodded to (or corrected to), people might upvote more than they should.
(I have also noticed that people don't stop upmodding even if the score is visible. I got 80 points yesterday on a one-line comment poking fun at Joel Spolsky. WTF?)
Just like when there is the delayed "reply" link that isn't showing, you can still reply to a comment by clicking the "link" link.
Edit: why is this being downmodded?
(I have also noticed that people don't stop upmodding even if the score is visible. I got 80 points yesterday on a one-line comment poking fun at Joel Spolsky. WTF?)
Just like when there is the delayed "reply" link that isn't showing, you can still reply to a comment by clicking the "link" link.
Edit: why is this being downmodded?
I notice that nobody here is playing devil's advocate, so allow me. What was wrong with the site beforehand? Sure the comments have a mob feel to them sometimes, but that's just something a social anything app has to deal with.
Over all, I like the comments on HN much more than say reddit or digg by far. Plus, I've never seen a community that responds so well to criticism as this one does.
Over all, I like the comments on HN much more than say reddit or digg by far. Plus, I've never seen a community that responds so well to criticism as this one does.
In long threads I skim based on scores. I try to give low comments at the bottom a quick read to vote on them, but one thing I definitely look for are long threads like this:
+5 -3 +4 -2 +4 0
That means someone is being generally rude or uninsightful, but the replies are good, so I'll just read the replies. If there's a long thread of a bunch of 1's, then two people are talking at each other both uninsightfully. If both people are voted up a bit, that means it's a good discussion and the whole thing is worth reading.
I'll give this a good trying out before forming an opinion either way, but the loss of skim-by-score ability is slowing me down just a touch right now. It'll be interesting to see how this turns out.
+5 -3 +4 -2 +4 0
That means someone is being generally rude or uninsightful, but the replies are good, so I'll just read the replies. If there's a long thread of a bunch of 1's, then two people are talking at each other both uninsightfully. If both people are voted up a bit, that means it's a good discussion and the whole thing is worth reading.
I'll give this a good trying out before forming an opinion either way, but the loss of skim-by-score ability is slowing me down just a touch right now. It'll be interesting to see how this turns out.
I'm seeing dots for some comments instead of the score. It looks like gray dots for comments with a high score and red dots for comments with a very high score. Doesn't this defeat the purpose of the change to some extent?
Edit: Sorry, it looks like this was explained after I had loaded this page. Still, this seems like a partial reversal. Also, this story is at #16 on the front page and really should be at #1 considering the magnitude of the change.
Oh and just one more question: I'm seeing this comment at the top of the page. Is that just how it looks for me, or is it that way for everyone because of the new ranking algorithm?
Another edit: From PG's explanation, I thought the gray dots were all turned red, but I still see gray dots for some comments (including one with a negative score).
Edit: Sorry, it looks like this was explained after I had loaded this page. Still, this seems like a partial reversal. Also, this story is at #16 on the front page and really should be at #1 considering the magnitude of the change.
Oh and just one more question: I'm seeing this comment at the top of the page. Is that just how it looks for me, or is it that way for everyone because of the new ranking algorithm?
Another edit: From PG's explanation, I thought the gray dots were all turned red, but I still see gray dots for some comments (including one with a negative score).
I think you can remove the word "by"
Interesting experiment. I thought I was going crazy all of a sudden.
A suggestion: Any chance the numbers could show later as the article gets older? I'd like to see how the votes resolved after some time. Or perhaps hide them only on front page articles?
A suggestion: Any chance the numbers could show later as the article gets older? I'd like to see how the votes resolved after some time. Or perhaps hide them only on front page articles?
I'm now experimenting with changing the function used to rank comments. I'm hopeful this will both help users find the good comments on a thread, and solve the problem of good but late comments getting stuck at the bottom of the page.
One PLUS of karma scores next to comments is that I know when to stop reading :)
If my personal interest + the comment score < arbitrary threshold, i quit
If my personal interest + the comment score < arbitrary threshold, i quit
If you had a Likert-type scale, this wouldn't be a problem. I would rate a post 7/10 regardless of what everyone else had rated it. But we just have ups and downs. I think we've all established a sense of what the "right" score is for a given comment. For example, a thoughtful, on-topic post probably has no upper limit for how many points it deserves, while a funny but off-topic post won't get an upvote if it's already got enough points.
Won't this lead us towards a bimodal distribution? Without the crucial knowledge of where the comment's score is, the votes will tend towards extremes: we can't collectively decide when the target comment is near the range that we find appropriate.
Won't this lead us towards a bimodal distribution? Without the crucial knowledge of where the comment's score is, the votes will tend towards extremes: we can't collectively decide when the target comment is near the range that we find appropriate.
The bandwagon effect (which is just ridiculous on reddit) occurs because top-rated comments stay at the top. Typical reddit comment at the top of a thread:
reddit_user 622 points
Glenn Beck is a wanker! He sniffs arses!
Weird suggestion: now hide the contributor too, avoiding the "vote pg up anyway"category of upvotes.
Then there's just a list of comments and one can vote them up or down on the content of the comment. Ok I did say it was weird.
EDIT: Some great counterpoints below. I agree with them. This is a bad idea.
Then there's just a list of comments and one can vote them up or down on the content of the comment. Ok I did say it was weird.
EDIT: Some great counterpoints below. I agree with them. This is a bad idea.
This new system saves me a lot of time by making HN less attractive.
It'd be interesting to see the results of the experiment - level of voting before and after, volatility of the voting before and after, rate of change of comment ordering, before and after. Are you planning on releasing some sort of aggregate data or publishing some sort of analysis?
Does anyone else find their eyes constantly scanning to try and find the score for each comment? I wonder if this means that I rely too heavily on the comment score? It's amazing how such a subtle change can reveal something more profound.
I'd try out rounded percentage scores.
Out of the total comment Karma alloted to all comments for an item, what proportion of this amount is contained within each comment?
The issue here is that upon voting, you couldn't change all displayed comment percentages at once, it'd have to change on page reload.
Also, it could be a rounded 1-10 scale.
-- edit: would be good to reveal any comment score upon voting: gives an incentive to vote and remain critical. It's also like a marker as to what comments interest you for future recall -- that's something new.
Out of the total comment Karma alloted to all comments for an item, what proportion of this amount is contained within each comment?
The issue here is that upon voting, you couldn't change all displayed comment percentages at once, it'd have to change on page reload.
Also, it could be a rounded 1-10 scale.
-- edit: would be good to reveal any comment score upon voting: gives an incentive to vote and remain critical. It's also like a marker as to what comments interest you for future recall -- that's something new.
This will definitely be interesting. While I think not having comment scores will have a somewhat limiting effect on groupthink voting, it will also probably have a counter-effect on pithy/silly one-liner comments by preventing people from noticing that no matter how humorous or witty the comment might have been it has already gathered as many votes as it deserves. The great karnak predicts a lot more early comments at the first level of two of comment threads being short witticisms that attempts to predict the herd and thereby gather a lot of early upvotes.
How about having the scores show up after a week (or X amount of time). Practically nobody votes on week-old posts, but I often use -- for example -- searchyc.com's average points per post to get a feel for a user's contributions.
That would balance the elimination of mob voting with being able to ascertain (historical) credibility.
That would balance the elimination of mob voting with being able to ascertain (historical) credibility.
Interesting. I still see some comments a light gray.
Also, like some others , I like know if something I've posted was particularly liked (or not).
And I've been skimming pages looking for the high-karma comments to try to dig out what are likely to be the most worthwhile comments. I suppose there's some feedback effect in place there, as popular comments simply get more attention (and so likely garner more up-votes votes), but there's pragmatic value in having a metric to selectively peruse a post with so much to read. (And, no, ZapRead isn't the answer :) )
Also, like some others , I like know if something I've posted was particularly liked (or not).
And I've been skimming pages looking for the high-karma comments to try to dig out what are likely to be the most worthwhile comments. I suppose there's some feedback effect in place there, as popular comments simply get more attention (and so likely garner more up-votes votes), but there's pragmatic value in having a metric to selectively peruse a post with so much to read. (And, no, ZapRead isn't the answer :) )
I love this idea, but it should really be initially implemented as a kind of A/B test.
Simply choose half of the UIDs at random and always show them the comment score, and don't show it to the other half. Track which comments are upvoted most when their score is visible versus when not (there are many other metrics to use as well). It should be pretty easy to determine how much groupthink plays into voting.
I suspect that most people only read (and therefore only vote on) already popular comments, so I suspect groupthink is fairly high here.
Simply choose half of the UIDs at random and always show them the comment score, and don't show it to the other half. Track which comments are upvoted most when their score is visible versus when not (there are many other metrics to use as well). It should be pretty easy to determine how much groupthink plays into voting.
I suspect that most people only read (and therefore only vote on) already popular comments, so I suspect groupthink is fairly high here.
I think only showing votes for comments and links once you've voted would be a better solution. It's what I'm trying with a side-project of mine.
I actually expect that posts may become more extremely negative than they used to. At some point, you become sorry for a person with a highly negative comment; or at least I do. Sometimes I upvote comments that are negative because I think that the downvotes are unreasonable. Now there's no way for others to provide a check on the immoral downvoting behavior of others.
Interesting idea, but now old threads like this, where people voted by modding up, don't make sense:
http://news.ycombinator.com/item?id=527681
http://news.ycombinator.com/item?id=527681
Voting still has all the same effects (on karma, and on position on the page), so I encourage users to keep doing it. The only difference is that comment scores aren't displayed in threads.