How Crowdsourcing Turned on Me(nautil.us)
nautil.us
How Crowdsourcing Turned on Me
http://nautil.us/issue/18/genius/how-crowdsourcing-turned-on-me
10 comments
Yeah that is what I immediately thought, that you would want to do some processing on the changes before presenting it to the users, and not make it live, just asking them to make a change and asking others to rate the change.
That would be a system that you would need a large number of users to manipulate.
The downside of this idea is that their website is clearly about an interactive experience where you move things around and get the feel of others doing the same thing, I dont know if that kind of thing would be compatible with y/our idea without a lot of tricks.
That would be a system that you would need a large number of users to manipulate.
The downside of this idea is that their website is clearly about an interactive experience where you move things around and get the feel of others doing the same thing, I dont know if that kind of thing would be compatible with y/our idea without a lot of tricks.
In other words, creation took 100 times as many moves and about 40 times longer than destruction.
So, yeah, pick your new engineering hires veeeery carefully.
So, yeah, pick your new engineering hires veeeery carefully.
Bad design. Anybody could mess up the job. In any large, anonymous group, there will be a certain number of assholes. Of course it didn't work. Ask anyone involved in running an IRC channel, or a MMORPG, or a factory.
Given the problem they were solving (reassembling images shredded into strips) they needed something where, when two strips were put together by a human, a program decided if they matched closely, and, if so, locked them together for further editing. That would constrain the workers to make forward progress. You might need an unlock capability if there was a false match, but that should take the cooperation of several people.
Given the problem they were solving (reassembling images shredded into strips) they needed something where, when two strips were put together by a human, a program decided if they matched closely, and, if so, locked them together for further editing. That would constrain the workers to make forward progress. You might need an unlock capability if there was a false match, but that should take the cooperation of several people.
>In any large, anonymous group, there will be a certain number of assholes. Of course it didn't work. Ask anyone involved in running an IRC channel, or an MMORPG, or a factory.
My favourite example: Twitch plays Pokemon
My favourite example: Twitch plays Pokemon
Or just versioning and the ability to revert, like Wikipedia has.
Really interesting article. I think we've only begun to scratch the surface of what's possible with crowdsourcing.
To me, the author's failure seems to be a straightforward one of design. The system allowed attackers to ruin everyone's work, and to do so in a much shorter time than the positive work took. It seems to me that it should be possible to design a system that doesn't have these properties. (Of course it's probably not trivial to do that without making it harder for honest people to participate. But that's a design challenge, not a fundamental flaw of crowdsourcing.)
To me, the author's failure seems to be a straightforward one of design. The system allowed attackers to ruin everyone's work, and to do so in a much shorter time than the positive work took. It seems to me that it should be possible to design a system that doesn't have these properties. (Of course it's probably not trivial to do that without making it harder for honest people to participate. But that's a design challenge, not a fundamental flaw of crowdsourcing.)
Anyone remember reading something very similar to this about a year ago? It might have been the same situation but I distinctly remember an article about crowdsourcing failing due to an attack.
So this is like a speeded up Wikipedia.
This was a good read, but the author is not above narcissism it seems.
The authors inability to empathise with "... and personally feel that crowdsourcing is basically cheating (and I’m not the only one that feels this way)." is something that makes me leery as well. That's an understandable viewpoint, how is this going to work with actually classified documents?
Still, like I said, good read.
He reminded me of the Luddites in the 19th century, who
destroyed the cotton- and wool-processing technology
that they feared would replace man with machine. Only in
this case, the concern was reversed: The attacker seemed
to dislike how crowdsourcing puts the collective
potential of humans above technology.
“artistic flare.” The words are now what I imagine when
trying to express the struggle between the creative,
collective mind that emerges online, and the dark,
almost eerie forces that antagonize all kinds of genius.
The authors inability to empathise with "... and personally feel that crowdsourcing is basically cheating (and I’m not the only one that feels this way)." is something that makes me leery as well. That's an understandable viewpoint, how is this going to work with actually classified documents?
Still, like I said, good read.
> In my experience, mobs like 4chan tend to descend on people who have high and mighty ideas of their actions.
In my experience, people who defend 4chan's actions like to victim-blame.
> The authors inability to empathise with "... and personally feel that crowdsourcing is basically cheating (and I’m not the only one that feels this way)." is something that makes me leery as well. That's an understandable viewpoint, how is this going to work with actually classified documents?
Actually this line (from the attacker) garnered the most attention from me. So the attacker felt that crowd-sourcing the solution was cheating - but maliciously attacking and sabotaging an opponent in a competition was just fine? This sense of entitlement is very typical of anti-social b-tard behaviour, and it shouldn't be encouraged.
Personally I find the tension between clever algorithms and crowd-sourcing very interesting. Applications like Foldit for protein folding produce some amazing results. Sabotaging valid solutions does nobody any favours, except in an extremely selfish and short-term way.
In my experience, people who defend 4chan's actions like to victim-blame.
> The authors inability to empathise with "... and personally feel that crowdsourcing is basically cheating (and I’m not the only one that feels this way)." is something that makes me leery as well. That's an understandable viewpoint, how is this going to work with actually classified documents?
Actually this line (from the attacker) garnered the most attention from me. So the attacker felt that crowd-sourcing the solution was cheating - but maliciously attacking and sabotaging an opponent in a competition was just fine? This sense of entitlement is very typical of anti-social b-tard behaviour, and it shouldn't be encouraged.
Personally I find the tension between clever algorithms and crowd-sourcing very interesting. Applications like Foldit for protein folding produce some amazing results. Sabotaging valid solutions does nobody any favours, except in an extremely selfish and short-term way.
>but maliciously attacking and sabotaging an opponent in a competition was just fine?
Empathise does not mean condone.
>This sense of entitlement
entitlement? Is someone acting entitled?
>very typical of anti-social b-tard behaviour,
nice...
>and it shouldn't be encouraged.
and it can't be stopped. It is a reality.
Empathise does not mean condone.
>This sense of entitlement
entitlement? Is someone acting entitled?
>very typical of anti-social b-tard behaviour,
nice...
>and it shouldn't be encouraged.
and it can't be stopped. It is a reality.
I don't think Darpa was looking for an algorithm to unshred documents, as the article states, they were assessing vulnerability of shredded documents. In that regard, this team's efforts were most important as they highlighted how an adversary doesn't even need a complex CV algorithm, all they need is a huge crowd and an incentive
That's an understandable viewpoint, how is this going to work with actually classified documents?
My thoughts exactly - the system is designed for the military, now imagine getting thousands of random people peeking at the documents.
My thoughts exactly - the system is designed for the military, now imagine getting thousands of random people peeking at the documents.
Fascinating stuff, not at all what I was expecting. Two things come to mind reading this.
1.) The crowdsourced and algorithmic solutions seem to be framed in contrast if not exclusive to each other here. I wonder if combining the two couldn't have helped avoid this:
>'However, the crowd was hopeless against a determined attacker. Before the first attack, our progress on the fourth puzzle had combined 39,299 moves by 342 users over more than 38 hours. Destroying all this progress required just 416 moves by one attacker in about an hour.'
I'm wondering if it's easier to create a 'bouncer' algorithm that detects and blocks/reverts aberrant behavior than to solve the core puzzle itself. If that problem is in fact easier - identifying 'hurtful' or unusual moves, could addressing it help the core solution by helping avoid non-malicious 'back sliding' too?
(I have no clue about these things.)
2.) The sort self-importance expressed by the attacker is just awful.
Unlike the author, I don't see it as a concern about "how crowdsourcing puts the collective potential of humans above technology". That almost sounds noble.
This impresses me as plain old envy and entitlement, an overgrown cousin to every "Why is this on HN?" or "Why is this on the front page? I submitted this before and no one voted for it." comment on HN. The whole thing is one big "should" of personal gratification.
1.) The crowdsourced and algorithmic solutions seem to be framed in contrast if not exclusive to each other here. I wonder if combining the two couldn't have helped avoid this:
>'However, the crowd was hopeless against a determined attacker. Before the first attack, our progress on the fourth puzzle had combined 39,299 moves by 342 users over more than 38 hours. Destroying all this progress required just 416 moves by one attacker in about an hour.'
I'm wondering if it's easier to create a 'bouncer' algorithm that detects and blocks/reverts aberrant behavior than to solve the core puzzle itself. If that problem is in fact easier - identifying 'hurtful' or unusual moves, could addressing it help the core solution by helping avoid non-malicious 'back sliding' too?
(I have no clue about these things.)
2.) The sort self-importance expressed by the attacker is just awful.
Unlike the author, I don't see it as a concern about "how crowdsourcing puts the collective potential of humans above technology". That almost sounds noble.
This impresses me as plain old envy and entitlement, an overgrown cousin to every "Why is this on HN?" or "Why is this on the front page? I submitted this before and no one voted for it." comment on HN. The whole thing is one big "should" of personal gratification.
> The sort self-importance expressed by the attacker is just awful
While I agree that instead of behaving maliciously, the other competitor should have lauded their simple yet effective solution, I can imagine the WTF he must have experienced upon discovering their site. I mean think about it, if your team has been spending god knows how long wracking your brains to come up with CV algorithms to solve this stuff, and then you realize the team in the lead isn't even using an algorithm! They are just using a huge amount of people to solve it! Now of course the right reaction to this should be to laugh wholeheartedly and ponder about how a motivated crowd can trump advanced technology.
We had a similar WTF moment during Google Code jam 2011. In the qualification rounds, there was a problem Goro Sort[1] and the way the question was written, it appeared to be a very tough problem. Later we discovered that one of our friends solved it with a VERY simple algorithm: just count the number of elements out of place! and it WORKS!! The good part was, instead of being jealous, we did laugh about it and still do
While I agree that instead of behaving maliciously, the other competitor should have lauded their simple yet effective solution, I can imagine the WTF he must have experienced upon discovering their site. I mean think about it, if your team has been spending god knows how long wracking your brains to come up with CV algorithms to solve this stuff, and then you realize the team in the lead isn't even using an algorithm! They are just using a huge amount of people to solve it! Now of course the right reaction to this should be to laugh wholeheartedly and ponder about how a motivated crowd can trump advanced technology.
We had a similar WTF moment during Google Code jam 2011. In the qualification rounds, there was a problem Goro Sort[1] and the way the question was written, it appeared to be a very tough problem. Later we discovered that one of our friends solved it with a VERY simple algorithm: just count the number of elements out of place! and it WORKS!! The good part was, instead of being jealous, we did laugh about it and still do
In my opinion, the attacker missed completely the point of the contest. Its aim wasn't to know who was the one who could come up with the most clever visual processing algorithm, but knowing which is the best way to solve that particular problem, and "Through computer-aid crowdsourcing" could very well be the best answer.
I really hope he wasn't in the winning team.
I really hope he wasn't in the winning team.
The ability to ban rogue users would likely have helped them immensely. An invitation-only system would probably have been of benefit as well.
Nothing is foolproof, obviously, but their users were working quickly. They only would have needed to keep malicious users away for a limited time in order to achieve success.
The author likely doesn't have much real-world experience with users or his team would have built this sort of capability into the software from the ground up. Any dev worth his or her salt knows to never trust any user.
Nothing is foolproof, obviously, but their users were working quickly. They only would have needed to keep malicious users away for a limited time in order to achieve success.
The author likely doesn't have much real-world experience with users or his team would have built this sort of capability into the software from the ground up. Any dev worth his or her salt knows to never trust any user.
And one of the crowd-sourcing examples, Wikipedia, has this built in functionality. Certain Wikipedia pages are under constant attack and diligent contributors do what they can to keep them accurate.
'In retrospect, it might have been foolish to assume that every member of an anonymous crowd would act according to our best interests.'
In my experience with crowdsourcing, most people actually will complete your task honestly and genuinely. Some may not understand it, but they generally just give back noise and cancel each other out. You'll almost always get a few malicious people though, even if their motivations aren't as straightforward as this case.
If you forget to account for this adversarial subset of folks, eventually you're gonna get got. Very cool story/use-case though.
In my experience with crowdsourcing, most people actually will complete your task honestly and genuinely. Some may not understand it, but they generally just give back noise and cancel each other out. You'll almost always get a few malicious people though, even if their motivations aren't as straightforward as this case.
If you forget to account for this adversarial subset of folks, eventually you're gonna get got. Very cool story/use-case though.
I believe Foursquare did something like this with regards to their recommendations. They'd ask questions about locations (e.g., pick a beautiful picture, is this place classy?, is it loud?, etc), and only when enough people agreed would they use the information in recommendations.