Voting Rings: How Insiders Game User Generated News(bothsidesofthetable.com)
bothsidesofthetable.com
Voting Rings: How Insiders Game User Generated News
http://www.bothsidesofthetable.com/2010/09/01/mafia-sourcing-the-insiders-game-of-user-powered-news/
8 comments
Why not simply mix some 'new' randomly into the front page at some ratio? If one or two of the stories in the front page were (different) new stories for each visitor, you'd solve the "front page or bust" problem.
You'd sacrifice a bit of "quality" from the front page, at the gain of a more variety and more exposure for new stories - entirely tunable by the ratio.
You'd sacrifice a bit of "quality" from the front page, at the gain of a more variety and more exposure for new stories - entirely tunable by the ratio.
Or display both in two columns. Frontpage on the left. New on the right.
Well, the idea would be that the casual viewer couldn't distinguish the 'new' from the front page and so would treat them the same.
Otherwise people will ignore new as they do now.
(Point score could be left off of the front page to help this).
Otherwise people will ignore new as they do now.
(Point score could be left off of the front page to help this).
I think the problem is not that people are wilfully ignoring it, rather it's just not prominent enough.
When I open HN, I am immediately seduced by an interesting headline (or point score) of a frontpage story. I don't have time to make a conscious decision to visit /newest. There's no way it's getting precedence over "12 ways to speed up your Erlang programs with NoSQL".
If new and the frontpage got equal treatment, then I'd be seduced by interesting headlines of frontpage and new stories equally. Meaning the good new stories would have a better chance of being upvoted.
I wouldn't like to see frontpage and new stories merged into the same list, since a great deal of new stories are spam or otherwise devoid of interest.
When I open HN, I am immediately seduced by an interesting headline (or point score) of a frontpage story. I don't have time to make a conscious decision to visit /newest. There's no way it's getting precedence over "12 ways to speed up your Erlang programs with NoSQL".
If new and the frontpage got equal treatment, then I'd be seduced by interesting headlines of frontpage and new stories equally. Meaning the good new stories would have a better chance of being upvoted.
I wouldn't like to see frontpage and new stories merged into the same list, since a great deal of new stories are spam or otherwise devoid of interest.
That would help to some extent but because of the different traffic ratios it would also significantly increase the pay-off for a spammer.
That's tunable again tho.
Not all front pages need some new mixed in - you can adjust both the amount of new and the proportion of 'contaminated' front pages.
Not all front pages need some new mixed in - you can adjust both the amount of new and the proportion of 'contaminated' front pages.
> How many people here ever click on "new"?
I don't know. But 'rather few' sounds accurate
> How many stories just disappear without hitting the front page?
About 90%. For 90% of that 90% that's fine. The other 10% are a loss.
> How many accounts need to be involved to make a particular story get to the front page?
Currently I'd peg that at somewhere between 4 and 5.
> Memory says that jacquesm mentioned once that he had turned down an offer of money to post certain articles from companies that were looking for good pr.
It wasn't for money it was in return for articles about my stuff.
I don't know. But 'rather few' sounds accurate
> How many stories just disappear without hitting the front page?
About 90%. For 90% of that 90% that's fine. The other 10% are a loss.
> How many accounts need to be involved to make a particular story get to the front page?
Currently I'd peg that at somewhere between 4 and 5.
> Memory says that jacquesm mentioned once that he had turned down an offer of money to post certain articles from companies that were looking for good pr.
It wasn't for money it was in return for articles about my stuff.
I was wondering about the consequences of introducing some kind of a supervisor system on HN. So practically every new/low-karma HN-member will be assigned to a high-karma HN-member. Every submission and/or voting needs then to be approved by a supervisor.
This would imply more work for high-karma members but I am sure they would love to contribute to the quality here on HN.
I would pity whatever high-karma HN-member got assigned me when I was new. I suspect the relationship would have irritated both sides very quickly, likely leading to me not having gotten involved in the first place.
Now that I'm a relatively high-karma member, I'm quite sure that I wouldn't want to have such a responsibility. I spend quite a bit of time here, sure, but in bursts. You can't rely on me to respond to anything within a day. And that lag time is inappropriate for how the site works.
Now that I'm a relatively high-karma member, I'm quite sure that I wouldn't want to have such a responsibility. I spend quite a bit of time here, sure, but in bursts. You can't rely on me to respond to anything within a day. And that lag time is inappropriate for how the site works.
There are for sure lots of potential technical methods to overcome the relationship problem. One of these would be assigning random high-karma users everytime someone submits a link.
Same goes for the time lag issue. You could assign only high-karma users who are currently browsing the site. In the end you could always opt-out.
Same goes for the time lag issue. You could assign only high-karma users who are currently browsing the site. In the end you could always opt-out.
But then we end up like Digg, right? And then who is watching the watchmen? If you give a super-user more weight then you're just increasing the potential power of a "corrupt" super-user.
I am not speaking about a longer period, you could 'graduate' in a couple of days. Apart from that one could easily detect if such a corruption takes place. For example a story that gets not approved but gets enough points and is submitted by someone else could be evidence for such a corruption. The one who has not approved the story could get minus points.
It's interesting to me that Kuro5hin in its heydey had relatively few problems with voting rings, despite having a purely voting-driven front page. There are probably some reasons involving community size and dynamics, but my guess is that the biggest one is the fact that the site was organized around submission of original articles to Kuro5hin, as opposed to submission of links. That raises the buy-in stakes beyond what people only interested in promotion are willing to spend time on, as well as lowering the potential gains from promotion (you got your article read, but directly on k5, not via traffic to any monetizable endeavor of yours). The fact that all votes were public may have had an effect as well, making any sort of collusion easier to spot.
It's also what made K5 in to a huge echo chamber.
I guess I didn't find that to be the case, any more than any other forum, at least. In what ways was K5 more of an echo chamber than, say, HN is?
If you write stuff for your 'own' audience of the K5 members, which will then write stuff as well for you to comment on and so on it's different than if 90% or more of the submitted articles is about stuff from the outside.
It brings in fresh blood, outside perspective and avoid inbreeding.
It brings in fresh blood, outside perspective and avoid inbreeding.
Ah, I guess that's the part I actually find much better. There's a certain community aspect where people are actually talking to each other, addressing an audience, that leads to much deeper engagement and discussions imo. That's also the feature of good mailing lists, BBSs, and Usenet groups I like best (though there are plenty of bad ones). The link-site thing seems too disjointed, and imo actually leads to more of an echo-chamber effect, because people end up having to pattern-match based on superficial things like headlines. So we get lots of random submissions that seem superficially about hackerdom or startups, but less sustained engagement. I mean, I personally find HN much more echo-chambery than K5 at its peak was. Perhaps personal preference.
Interesting. You've convinced me to re-visit K5 to see what it's like there now.
Haha, I wouldn't suggest it. I still post there, but it's basically a dive bar of old regulars and some old trolls (who are now a bigger proportion, but have been around long enough that they're sort of endearing, like the local drunk). Very little technical content, and nobody ever writes/submits stories any more, although there is occasionally technical stuff in the diaries (Luke from prgmr.com still posts some stuff there). I think early 2000s was its peak, it was on life support by around 2006, and from 2008-present it's been the walking dead.
An interesting question (to me) is what killed it. One might be that Rusty Foster increasingly lost interest in too much active sysadminning, and eventually added a $5 registration fee, which he thought would raise the quality of the userbase, but actually ended up completely killing any new registration. Another might be the rise of blogs. Although I like the submit-to-K5 model, blogs do make it a lot easier to just post on your personal blog and link somewhere if you prefer. I think there are especially economic incentives to write good content on your own blog rather than on a community site these days (you can make a little money from even a personal blog, get pagerank/prestige from it if it's attached to other projects, etc.).
Of course, that's exactly the part I'm ambivalent about, even though I do it too. I think something's lost when you go from 100 people discussing in a community (whether it's a site or a mailing list) to 100 people each trying to build their personal brands.
An interesting question (to me) is what killed it. One might be that Rusty Foster increasingly lost interest in too much active sysadminning, and eventually added a $5 registration fee, which he thought would raise the quality of the userbase, but actually ended up completely killing any new registration. Another might be the rise of blogs. Although I like the submit-to-K5 model, blogs do make it a lot easier to just post on your personal blog and link somewhere if you prefer. I think there are especially economic incentives to write good content on your own blog rather than on a community site these days (you can make a little money from even a personal blog, get pagerank/prestige from it if it's attached to other projects, etc.).
Of course, that's exactly the part I'm ambivalent about, even though I do it too. I think something's lost when you go from 100 people discussing in a community (whether it's a site or a mailing list) to 100 people each trying to build their personal brands.
So, I did take a look. It's mostly dead. Pity!
HN may have voting rings, but it still works because it's a niche site, and hackers have strong Internet BS detectors that counter spam/phishing/linkjacking/other mischief relatively effectively. People on HN may be killing time, but the majority still have some quality standards for what they want to read. Gaming HN isn't worth as much because A) there's fewer people, and B) you have to work harder to game it. Other niche sites can easily share this quality.
But for a site like Digg that wants to be the biggest social news site, it's just a race to the bottom. Gaming Digg just requires a tabloid-editor mentality where you figure out the quickest thing that will grab peoples attention and then boost it to the front page.
But for a site like Digg that wants to be the biggest social news site, it's just a race to the bottom. Gaming Digg just requires a tabloid-editor mentality where you figure out the quickest thing that will grab peoples attention and then boost it to the front page.
YC is already one giant mafia, if you'd like to think of it that way. I'm sure YC classes vote each others' stuff up.
I'm wondering what the recently implemented voting ring detection reveals. Are they all clustered by YC class? Are there overlaps? Or is there an outside group moving the articles up?
I'm wondering what the recently implemented voting ring detection reveals. Are they all clustered by YC class? Are there overlaps? Or is there an outside group moving the articles up?
When you build a site with user submitted content and voting, you have two immediate problems you need to solve before you can even get started:
You really need to bake something in to handle it from the word go.
- How are you going to deal with spam
- How are you going to deal with gaming
You really need to bake something in to handle it from the word go.
What you need to truly defeat this sort of spam is personalization, AKA recommendation,
If the top news stories are personalized to your interests, it is more difficult to game the system.
In particular, consider this scenario: I get a news result from a voting ring, and it ranks high because they all voted. But then I downvote the article because it is not interesting to me, Afterwards, the voting ring's weight is diminished in my personalized news feed. Hence, they cannot artificially rank articles for you,
I would argue that without some personalization, it is impossible to 100% avoid spam. Because if you treat there as being one objective lense, it is hard to discriminate between legitimate overlapping interests and colluders who share their authority.
If the top news stories are personalized to your interests, it is more difficult to game the system.
In particular, consider this scenario: I get a news result from a voting ring, and it ranks high because they all voted. But then I downvote the article because it is not interesting to me, Afterwards, the voting ring's weight is diminished in my personalized news feed. Hence, they cannot artificially rank articles for you,
I would argue that without some personalization, it is impossible to 100% avoid spam. Because if you treat there as being one objective lense, it is hard to discriminate between legitimate overlapping interests and colluders who share their authority.
Indeed, though that would only fix it for the small percentage of users who are (a.) logged in, and (b.) vote things down.
It wouldn't work on me, for example, because I hardly ever vote on anything here. In the 3 years I've been around I can't imagine I've actually voted down more than a half dozen times.
It wouldn't work on me, for example, because I hardly ever vote on anything here. In the 3 years I've been around I can't imagine I've actually voted down more than a half dozen times.
Unless you rated in whether or not you read the article/comments, in which case it might still work.
And of course, if downmod meant more "hide this sort of thing from me" than "I don't think this should be on HN", maybe you'd downmod more...
(But this starts showing the downside to such personalization - it's a lot more effort server-side, and it dilutes a single community)
And of course, if downmod meant more "hide this sort of thing from me" than "I don't think this should be on HN", maybe you'd downmod more...
(But this starts showing the downside to such personalization - it's a lot more effort server-side, and it dilutes a single community)
You don't have to be logged in - just give them a long-term cookie when they show up on your site and it still works.
And if you aren't voting anything down, you wouldn't be the kind of person who would need such a website in the first place.
And if you aren't voting anything down, you wouldn't be the kind of person who would need such a website in the first place.
You lost me there. You're talking about this on a site that doesn't even allow its users to downvote stories. And you're talking to a power user of that site, who just told you he doesn't ever downvote things. HN is a culture of voting up things that are good, and, rarely, downvoting things that are distracting from the conversation. That's one of the reasons it's so good.
As to cookies for non-logged in users, they don't really help anything. Sure, you could consider every clickthru as a 'vote' for personalization purposes, but it would likely just fill a user's feed with poorly titled things and clickbait. Probably more than half the articles I click through on get closed immediately without reading. I wouldn't want to optimize my view to include more of that.
You lost me there. You're talking about this on a site that doesn't even allow its users to downvote stories. And you're talking to a power user of that site, who just told you he doesn't ever downvote things. HN is a culture of voting up things that are good, and, rarely, downvoting things that are distracting from the conversation. That's one of the reasons it's so good.
As to cookies for non-logged in users, they don't really help anything. Sure, you could consider every clickthru as a 'vote' for personalization purposes, but it would likely just fill a user's feed with poorly titled things and clickbait. Probably more than half the articles I click through on get closed immediately without reading. I wouldn't want to optimize my view to include more of that.
Okay we try again. If he didn't wish he could downvote stuff, he hasn't got a need for a service that takes his downvote into account.
The long term cookie isn't really that important for vote counting perspectives but to be able to find out whom to serve the personalized stuff to. It is essentially no different from hn which also sets a cookie when you login, except that if you erase the cookie, you can get it set again when you log in, which wouldn't happen in my example.
The long term cookie isn't really that important for vote counting perspectives but to be able to find out whom to serve the personalized stuff to. It is essentially no different from hn which also sets a cookie when you login, except that if you erase the cookie, you can get it set again when you log in, which wouldn't happen in my example.
>And you're talking to a power user of that site, who just told you he doesn't ever downvote things.
As an aside, what makes you a "power user"?
As an aside, what makes you a "power user"?
Same thing that makes you one, brother. 1000 reputation points and the equivalent of $10k in otherwise billable hours squandered on this site.
I'm going to be lording it over the hoi-poloi now, "I'm a power user".
It's quite possible to generate recommendations solely from upvotes.
I intend to spend next year building just this. The problem I have, which I suspect all similar efforts will face, is how to bootstrap a network like this, especially when your competitors are the social news sites where all your future users hang out.
I think this is a good time to launch a new news aggregation site. Reddit has "jumped the shark" so to speak and I think there is a large chunk of displaced users looking for an outlet. That brings up the third problem with these sites, how to gracefully get bigger without quality dropping. Of course Reddit did in a sense solve this problem by allowing Reddit itself to be the next solace for displaced people, they just jump down into a deeper/more obscure subreddits.
I'd bet there are voting rings on HN. Perhaps not as bad as was on Digg. But I'm sure it's here. PG made a comment about it the other day, offhand, in something about getting some code/idea help about fighting it.
The comment was on Twitter, I think: http://twitter.com/paulg/status/22551773068
Makes me want to look at the HN source, as I find this problem very interesting. Or hoping for a "plan for voting rings" article :-)
Makes me want to look at the HN source, as I find this problem very interesting. Or hoping for a "plan for voting rings" article :-)
Since many people have dynamic IPs, I think tracking geolocation patterns of the first 10-ish votes would be enough to distinguish suspect cases and eventually put them 'on-hold' (HN dosen't really need to get stories on the front page as soon as possible, it's not exactly a breaking news site).
Since people have to be logged in to vote, there is a much easier solution:
Count votes from new users lower.
And count early votes from users often voting on articles posted by a particular person lower.
It can be refined quite a bit...
Count votes from new users lower.
And count early votes from users often voting on articles posted by a particular person lower.
It can be refined quite a bit...
True, but the problem with voting rings is that links may be submitted by different participants to the ring each time, and if they all have genuine high karma, that is difficult to track down.
There's a fine line between a couple dozen guys on IRC with similar interests and an actual voting ring. I'm not sure where one concept stops and another starts. I guess if you demanded money for voting, that'd obviously be bad. But if you IM'ed a couple dozen friends about a cool new link you saw? It gets fuzzier.
I have noticed voting trends by time of day on HN. I've also noticed what appears to be auto upvoting and downvoting based on the submitter. But I'm just an outsider, so what do I know? Up until recently, I have always assumed such activity to be more demographically-based. I am slowly beginning to have my doubts, though.
The interesting point about HN is, at the end of the day, this is PGs site. Maybe he decides he wants articles about squirrels. So we get articles about squirrels. Maybe he decides site X is trollish. So we get no stuff from site X.
It's been called a "benign dictatorship". Meh. Sounds a bit like "jumbo shrimp" to me. Even though HN isn't what it used to be, I still visit.
I have noticed voting trends by time of day on HN. I've also noticed what appears to be auto upvoting and downvoting based on the submitter. But I'm just an outsider, so what do I know? Up until recently, I have always assumed such activity to be more demographically-based. I am slowly beginning to have my doubts, though.
The interesting point about HN is, at the end of the day, this is PGs site. Maybe he decides he wants articles about squirrels. So we get articles about squirrels. Maybe he decides site X is trollish. So we get no stuff from site X.
It's been called a "benign dictatorship". Meh. Sounds a bit like "jumbo shrimp" to me. Even though HN isn't what it used to be, I still visit.
oooooh, i'm so scared... but excuse me, I've got to make a few 1000 electric sheep so I can get some traffic to my blog...
I'm sure that the answers respectively are, "rather few", "the vast majority" and "not many". And there you have the raw ingredients for a voting ring to form. As a matter of principle I would never be involved with such, and I have a natural dislike for anyone that I think is. But the ingredients are there.
More than that, you still have a popularity effect. People who vote on new articles have to prioritize what they read in some fashion. Speaking personally, it is natural to choose to look at articles with interesting titles, or which are submitted by someone you recognize as being a consistent source of good content. Therefore if someone like patio11 submits an article, it has a significant advantage. As does anything involving YC. (Of course promoting YC is the whole point of this site, so I don't mind that at all. And I'm guilty at looking at content from people I trust before content from people I've never heard of.)
Memory says that jacquesm mentioned once that he had turned down an offer of money to post certain articles from companies that were looking for good pr. (I don't swear that I correctly remember who it was.) He may have turned them down, but I'm sure they asked again, and if they asked around enough I'm sure that they found a taker. And if so, then you have the dark side here on HN.