The Dollhouse Mafia, or "Don't Display Negative Karma"(buildingreputation.com)
buildingreputation.com
The Dollhouse Mafia, or "Don't Display Negative Karma"
http://buildingreputation.com/writings/2009/10/the_dollhouse_mafia_or_why_to.html
31 comments
I don't know that it's going to happen anytime soon, but I can see exchanges between karma and real currency eventually leading to a reputational economy. See Doctorow's Down and Out in the Magic Kingdom. http://craphound.com/down/download.php I wish that pg would start allowing karma transfers here so we could start experimenting and I could cash in all of my stupid points, but that experiment is not one of his goals.
For a while I've told myself that I'd like to auction this username for charity, mostly just to see what happens, though I like the idea of converting my years of activity here into a charitable donation.
On Hacker News, karma really does just measure activity. My point is that it is theoretically possible to give karma more meaning than it has today.
For a while I've told myself that I'd like to auction this username for charity, mostly just to see what happens, though I like the idea of converting my years of activity here into a charitable donation.
On Hacker News, karma really does just measure activity. My point is that it is theoretically possible to give karma more meaning than it has today.
You might be interested in the book "Accelerando" < http://www.amazon.com/exec/obidos/ASIN/0441014151/charlieswe... >, as one of its plot points is that the basic unit of exchange in the economy changes from money to reputation.
It is also available online, with full consent of the relevant rights owners: http://www.antipope.org/charlie/accelerando/
My point is that it is theoretically possible to give karma more meaning than it has today.
... or less, if they were to implement a karma trading system.
... or less, if they were to implement a karma trading system.
Wait, how would karma transfers work? What would be the point?
A likely implementation seems a box where you type the username and points for the transfer recipient. I believe this was discussed a while ago, and someone mentioned that they would do programming work for karma.
I have the karma and it doesn't mean anything to me, but it means something to some people that don't have it. There are people out there that would trade real world goods and services for Hacker News karma.
The point would just be as an experiment. I'm not expecting anything good would happen immediately; we'd probably see more abuse than anything.
I have the karma and it doesn't mean anything to me, but it means something to some people that don't have it. There are people out there that would trade real world goods and services for Hacker News karma.
The point would just be as an experiment. I'm not expecting anything good would happen immediately; we'd probably see more abuse than anything.
But if you could buy karma, it would stop meaning what it supposedly means now, that you said some combination of allegedly intelligent things.
I mostly agree with you, but it's not hard to create a karma system that isn't just a reward for hyperactivity. For example, Hacker News could change its system so that nobody got a point simply for posting a comment.
But however fine-tuned the system, no point system is immune from gaming, and no-one can expect points to be any more than vague clue as to someone's true worth as a participant in the community.
But however fine-tuned the system, no point system is immune from gaming, and no-one can expect points to be any more than vague clue as to someone's true worth as a participant in the community.
"For example, Hacker News could change its system so that nobody got a point simply for posting a comment."
You don't get a point simply for posting a comment: all points come from upvotes - downvotes. Initial "1 point" label is a purely cosmetic thing.
You don't get a point simply for posting a comment: all points come from upvotes - downvotes. Initial "1 point" label is a purely cosmetic thing.
Aha, I didn't realise. So HN is already an example of a karma system that doesn't reward hyperactivity.
I agree that voting systems on the web end up favoring the most persistent, rather than the most worthy or highest quality. This is something that I've seen on Wikipedia and Digg. And to an extent with the trolls on Kuro5hin.
I think moderation is important, but that it should not be the central aspect of user interaction.
I think moderation is important, but that it should not be the central aspect of user interaction.
Karma/Reputation systems on website are to reward a desired behavior. Part of HN's value is it's active useful conversation (vs /.,digg etc). So it makes sense to give the community the tools to reward what they want and create positive self feedback. (Also like the other reply, simply posting doesn't give you karma)
There are two problems with what you say. First, a large number of posts with a small number of upvotes is still valued more by HN than a small number of well-respected comments. Second, moderation systems tend to be inflationary, which means that the point value of a given comment has more to do with how many people read the thread than anything else.
Slashdot's point cap is the obvious solution, but even there comments tend to lean heavily towards +5 on well read threads.
Slashdot's point cap is the obvious solution, but even there comments tend to lean heavily towards +5 on well read threads.
I don't think they are necessarily problems.
Re: first problem: I think a large number of fairly good comments probably IS a more valuable contribution than a small number of very good comments.
The second problem is more, er, problematic, but you could argue that good comments are popular articles deserve more karma, since these are comments on topics which people are really interested in – i.e. valuable insights on hot topics benefit more people than valuable insights on obscure topics, and are therefore more valuable contributions.
Re: first problem: I think a large number of fairly good comments probably IS a more valuable contribution than a small number of very good comments.
The second problem is more, er, problematic, but you could argue that good comments are popular articles deserve more karma, since these are comments on topics which people are really interested in – i.e. valuable insights on hot topics benefit more people than valuable insights on obscure topics, and are therefore more valuable contributions.
One way around the mafia problem is a kind of normalisation, though it has its own flaws. A negative vote from someone who always gives negative votes counts for less than a negative vote from someone who generally gives positive votes. You can add in thresholding and voting ring detection for some fun, resulting in normalisation of their opinion to some degree.
There's also normalisation on the other end, how many of the total votes are negative or positive - but this doesn't really work unless there's a transaction cost, as fake accounts can shill the voting.
There's also normalisation on the other end, how many of the total votes are negative or positive - but this doesn't really work unless there's a transaction cost, as fake accounts can shill the voting.
One of my ideas about moderation and rating systems is that the act of moderation tells us more about the person doing the moderating than it does about the story/post/person being moderated.
If we think about moderation as relational rather than objective or absolute or aggregating, we can treat the user's moderation activity as their particular web of relations to various other objects (stories/posts/users).
We could then drop all these relationships into an algorithm built to examine large matrices of relationships, say... PageRank. I wrote a tl;dr piece on moderation and online communities here: http://www.kuro5hin.org/story/2009/3/12/33338/3000
If we think about moderation as relational rather than objective or absolute or aggregating, we can treat the user's moderation activity as their particular web of relations to various other objects (stories/posts/users).
We could then drop all these relationships into an algorithm built to examine large matrices of relationships, say... PageRank. I wrote a tl;dr piece on moderation and online communities here: http://www.kuro5hin.org/story/2009/3/12/33338/3000
Funny, I've also been doing some thinking about moderation and implicit/explicit voting recently. Fascinating topic. Here's my piece: http://www.trendpreneur.com/online/link-voting-real-time-res...
Advogato's trust metric fleshed out this sort of idea. I wonder why it (and related work) hasn't seen more use.
Everyone has a different idea of what karma should be for. Conspiracy theorists trust other conspiracy theorists and loathe anyone with a mundane explanation. Bicyclists like each other more than car owners.
So what you need is to give everyone their own karma table to rate other users. You arrive, and you are neutral towards everyone else. After a while, you vote some people up and some people down. As you establish what your own standards are, you are establishing your own weighting of different social behaviors. Your weightings are likely to correlate strongly with those of other users - most likely the majority group, but maybe you have a lot in common with an established minority crowd (eg UFO 'believers'). New encounters can then be assigned provisional karma scores based on the average scores handed out by the group with whom your own weightings correlate most strongly. Thus, if you are a UFO believer you will see provisional negative karma for the science geek who loves to debunk UFO theories, while if you are yourself a science geek you'll see provisional negative karma for UFO believers in accordance with the fact that your karma votes correlate strongly with similarly-minded people.
Characterizations of UFO believers, scientists etc. are based on internet stereotypes, not responsible for lost ego or damaged feelings
Obviously, a system like this with N users has N^2 karma scores and the cost of administering the karma system increases exponentially compared to linear user growth. But there are a lot of algorithms you could use to flatten this, since in practice there will be groups whose membership follows a power-law distribution, rather than purely atomistic noisy preferences. Another possibility is a local (client-side) karma table with p2p lookup.
Another downside is that groups will tend toward partisanship since there will not be any universal penalties for most unpopular views (obviously, some views are so extreme that they'll never gain any social traction); even minority thinkers will have a circle of friends who share an unpopular belief and provide mutual support (as can be seen across the internet). But this system accommodate pluralities, whereas attempts at universal karma result either in relative homogeneity (vast #s of echo chamber websites) or else the complete breakdown and abandonment of the karma concept, with attendant signal degradation (eg 4chan/b/).
So what you need is to give everyone their own karma table to rate other users. You arrive, and you are neutral towards everyone else. After a while, you vote some people up and some people down. As you establish what your own standards are, you are establishing your own weighting of different social behaviors. Your weightings are likely to correlate strongly with those of other users - most likely the majority group, but maybe you have a lot in common with an established minority crowd (eg UFO 'believers'). New encounters can then be assigned provisional karma scores based on the average scores handed out by the group with whom your own weightings correlate most strongly. Thus, if you are a UFO believer you will see provisional negative karma for the science geek who loves to debunk UFO theories, while if you are yourself a science geek you'll see provisional negative karma for UFO believers in accordance with the fact that your karma votes correlate strongly with similarly-minded people.
Characterizations of UFO believers, scientists etc. are based on internet stereotypes, not responsible for lost ego or damaged feelings
Obviously, a system like this with N users has N^2 karma scores and the cost of administering the karma system increases exponentially compared to linear user growth. But there are a lot of algorithms you could use to flatten this, since in practice there will be groups whose membership follows a power-law distribution, rather than purely atomistic noisy preferences. Another possibility is a local (client-side) karma table with p2p lookup.
Another downside is that groups will tend toward partisanship since there will not be any universal penalties for most unpopular views (obviously, some views are so extreme that they'll never gain any social traction); even minority thinkers will have a circle of friends who share an unpopular belief and provide mutual support (as can be seen across the internet). But this system accommodate pluralities, whereas attempts at universal karma result either in relative homogeneity (vast #s of echo chamber websites) or else the complete breakdown and abandonment of the karma concept, with attendant signal degradation (eg 4chan/b/).
> Obviously, a system like this with N users has N^2 karma scores and the cost of administering the karma system increases exponentially compared to linear user growth.
Do you imply that N^2 is exponential? Or is there another way to understand your sentence.
The rest of your post is quite insightful.
Do you imply that N^2 is exponential? Or is there another way to understand your sentence.
The rest of your post is quite insightful.
The article outlines only one major reason for karma-based systems.
People can get addicted to their rating, creating "karma whores" and the like. This is nothing that probably hasn't been discussed here and on other communities.
Its a balance between the web-site operators wanting people to come back (what is my karma now after that awesome comment?), and pissing people off due to the problems karma systems introduce along with a general decline in community quality (Reddit anyone?).
Is it too much to ask for people to just have online conversations for the sake of getting involved? The only real CLEAN motivation is intrinsic, but that says a lot more about human nature than online rating systems.
People can get addicted to their rating, creating "karma whores" and the like. This is nothing that probably hasn't been discussed here and on other communities.
Its a balance between the web-site operators wanting people to come back (what is my karma now after that awesome comment?), and pissing people off due to the problems karma systems introduce along with a general decline in community quality (Reddit anyone?).
Is it too much to ask for people to just have online conversations for the sake of getting involved? The only real CLEAN motivation is intrinsic, but that says a lot more about human nature than online rating systems.
People can get addicted to their rating, creating "karma whores" and the like.
A counterpoint to this: If a karma system reliably rewards beneficial behavior, "karma whores" can potentially create a lot of real value for more casual members of the community (at the cost of possibly some irritation from active but not addicted members). This may not be possible in all communities, of course, but here's at least one example: http://stackoverflow.com/users/22656/jon-skeet
A counterpoint to this: If a karma system reliably rewards beneficial behavior, "karma whores" can potentially create a lot of real value for more casual members of the community (at the cost of possibly some irritation from active but not addicted members). This may not be possible in all communities, of course, but here's at least one example: http://stackoverflow.com/users/22656/jon-skeet
you've just reinvented the invisible hand concept.
for the record I agree completely. checks and balances on negative behaviors is stupid and has never worked. what you need to do is set up the incentives so that everyone's goals are complementary anyway. then the users police themselves because everyone is benefiting. you've turned a zero-sum game (status) into positive sum.
for the record I agree completely. checks and balances on negative behaviors is stupid and has never worked. what you need to do is set up the incentives so that everyone's goals are complementary anyway. then the users police themselves because everyone is benefiting. you've turned a zero-sum game (status) into positive sum.
The problem is that "karma whores" gravitate towards actions that are merely structurally beneficial without regard to actual value. It's cargo-cult commenting.
I agree, which is why I support downplaying the egoistic elements of online communities. Hiding usernames where possible, hiding raw karma scores or post ratings, etc. The less systemic/administrative information plastered all over the place, the more the users will (hopefully) focus on content.
On the other hand, I think the HN karma system has worked well to maintain focus in the site; lame attempts at humor get smacked hard, as do "+1" posts, coarse language, ad hominem arguments, etc. If the post rating were hidden, you wouldn't know if a down-voted post was due to one person disagreeing with you or a major breach of etiquette.
(Of course, in several ways, karma is a poor motivator for behavior that is valuable to the community.)
(Of course, in several ways, karma is a poor motivator for behavior that is valuable to the community.)
Yes, I'd say Hacker News has done an unbelievably good job of keeping things good around here. I am your typical Digg->Reddit->Hacker News migrator (I stayed on Reddit WAY too long...).
If anything, I say that here at HN we use "karma" as more of an indicator of RELEVANCY. Upmod comments that are relevant to the discussion, and leave it alone otherwise (very similar to the conclusion of the linked article).
If anything, I say that here at HN we use "karma" as more of an indicator of RELEVANCY. Upmod comments that are relevant to the discussion, and leave it alone otherwise (very similar to the conclusion of the linked article).
I agree. I think one would have to have some sort of automatic moderation in place. Something like the Robot 9000 built for the xkcd IRC channel that moderates on the basis of originality (such that duplicate noise comments get chatters a temporary muting). Perhaps one-liners below a certain length could be posted at a score of 0 instead of 1. Other automatic wordfilters could also aid in downplaying the user-as-judge-and-jury aspect of moderation.
Personally I think it would be better to only allow upvote, and reserve flagging for inappropriate comments. I don't feel it's necessary to downvote anything except if it's racist, offensive, or in any other ways inappropriate.
I mean I saw a person who was obviously not an English speaker post a hard to understand comment. He was voted down a couple of times but I can't understand why. His comment wasn't offensive. He may have had some misunderstanding and some trouble communicating but I don't think he had any negative intent. It seems in these cases negative voting serves no purpose. If you just want good comments highlighted, upvoting should be enough. Flagging can be used for innapropriate stuff.
I mean I saw a person who was obviously not an English speaker post a hard to understand comment. He was voted down a couple of times but I can't understand why. His comment wasn't offensive. He may have had some misunderstanding and some trouble communicating but I don't think he had any negative intent. It seems in these cases negative voting serves no purpose. If you just want good comments highlighted, upvoting should be enough. Flagging can be used for innapropriate stuff.
What about using a small graph instead of a sum total to show karma earning/loss over time?
Just about any other karma system winds up measuring how hyper-active someone is online. My karma score tells you nothing about me at all except that I'm here a lot and talkative.
This can be good thing. Much real life bad or good reputation is unjustified.