What If Scientists Didn’t Compete?(tierneylab.blogs.nytimes.com)
tierneylab.blogs.nytimes.com
What If Scientists Didn’t Compete?
http://tierneylab.blogs.nytimes.com/2009/04/30/what-if-scientists-didnt-compete/
13 comments
When your entire research program is based on your having to convince your funding agency to give the bucks to you and not to the other guy, you create competition and moreover you are rewarding the people who are the most competitive.
This seems ridiculously inefficient. The belief that two people spending money and effort to keep at odds with one another is beneficial and furthers advancement more rapidly than collaboration seems broken. With all of that waste, I can't understand how anyone at the funding agency can say they are getting the most return on investment following this strategy. A more collaborative model would give them the dual benefit of not having all of this in-fighting waste, while at the same time benefiting from the super-linear growth in results by having two people bang heads together. I sense a sacred cow here.
As for how it can be done, what about allowing funding agencies to grant money to something akin to a research project trusts or foundations; something that spans multiple individuals and even multiple institutions, so the lines don't have to be drawn so sharply on who can participate and who can't. Something like the foundation model several OSS projects have adopted, perhaps.
This seems ridiculously inefficient. The belief that two people spending money and effort to keep at odds with one another is beneficial and furthers advancement more rapidly than collaboration seems broken. With all of that waste, I can't understand how anyone at the funding agency can say they are getting the most return on investment following this strategy. A more collaborative model would give them the dual benefit of not having all of this in-fighting waste, while at the same time benefiting from the super-linear growth in results by having two people bang heads together. I sense a sacred cow here.
As for how it can be done, what about allowing funding agencies to grant money to something akin to a research project trusts or foundations; something that spans multiple individuals and even multiple institutions, so the lines don't have to be drawn so sharply on who can participate and who can't. Something like the foundation model several OSS projects have adopted, perhaps.
Inefficient doesn't begin to describe it.
The thing to remember is there's no money in OSS to first approximation - and you don't need money to write OSS. The problem is, you need a lot of money to do science, at least certain kinds of science.
So here is how it works in practice. Say you have an idea on how to make a breakthrough on topic X. You figure you need to pay a research assistant for a year, or buy a week on some instrument, or whatever. Either way, say you need 50K.
Case 1: You are the only person who wants 50K to do X, you need to convince the agency that X needs doing. This is an advantageous situation to you and depends on only you having the X idea, so you become very closed source about it. This is sub-optimal because it delays people looking into X. Also it is a turnoff to the kind of people who are interested in science and not skullduggery.
Case 2. X is in fact a compelling avenue of research, so three people apply to the funding agency for 50K. This is the situation I was describing upthread - now these people have to compete, and prove that they are the best positioned to get an an answer to X if only you give them the 50K. This is suboptimal because by definition 2 out of 3 applicants wasted their time applying for X. Think of the comparison with YC - the startups who get turned down by YC have other options for getting money; moreover they only have the one project they need to worry about getting funded. Whereas a research scientist typically only has one source of money, but many lines of inquiry. This is why scientific seniority often means the death of research time, with people spending large amounts of time on the scatter shot approach - apply to the funding agency for 8-9 projects in the hope that 1-2 will get funded, because if the agency doesn't fund you for something you get to lose your staff and then you are nowhere.
The situation becomes worse if what you need is not 50K but 50M, say to build a new instrument. Now your agency can't afford that at all, so the only way to get the money is to form an inter-agency (typically international) collaboration. Now the real inefficiency nightmare comes in; because what the other agencies buy with their money is the right of their people to work on the project. So not only the project becomes a sprawling nightmare, but the rules typically are than if Timbuktu brings 10M out of 50M to the table, they get to do 20% of the work on building the instrument - quite irrespective of whether they are the most qualified people to be doing so.
As I said, I do not know what the answer is. The reason the situation is like that is an honest attempt by the agencies to be responsible with taxpayer money; the money is not infinite; they have to decide who to give the money to; and they feel, for reasons that are obvious, that competition is the best way to make sure that the money goes to the best person. I don't think anybody working in science would claim the system is perfect, but other workable solutions are not forthcoming.
The thing to remember is there's no money in OSS to first approximation - and you don't need money to write OSS. The problem is, you need a lot of money to do science, at least certain kinds of science.
So here is how it works in practice. Say you have an idea on how to make a breakthrough on topic X. You figure you need to pay a research assistant for a year, or buy a week on some instrument, or whatever. Either way, say you need 50K.
Case 1: You are the only person who wants 50K to do X, you need to convince the agency that X needs doing. This is an advantageous situation to you and depends on only you having the X idea, so you become very closed source about it. This is sub-optimal because it delays people looking into X. Also it is a turnoff to the kind of people who are interested in science and not skullduggery.
Case 2. X is in fact a compelling avenue of research, so three people apply to the funding agency for 50K. This is the situation I was describing upthread - now these people have to compete, and prove that they are the best positioned to get an an answer to X if only you give them the 50K. This is suboptimal because by definition 2 out of 3 applicants wasted their time applying for X. Think of the comparison with YC - the startups who get turned down by YC have other options for getting money; moreover they only have the one project they need to worry about getting funded. Whereas a research scientist typically only has one source of money, but many lines of inquiry. This is why scientific seniority often means the death of research time, with people spending large amounts of time on the scatter shot approach - apply to the funding agency for 8-9 projects in the hope that 1-2 will get funded, because if the agency doesn't fund you for something you get to lose your staff and then you are nowhere.
The situation becomes worse if what you need is not 50K but 50M, say to build a new instrument. Now your agency can't afford that at all, so the only way to get the money is to form an inter-agency (typically international) collaboration. Now the real inefficiency nightmare comes in; because what the other agencies buy with their money is the right of their people to work on the project. So not only the project becomes a sprawling nightmare, but the rules typically are than if Timbuktu brings 10M out of 50M to the table, they get to do 20% of the work on building the instrument - quite irrespective of whether they are the most qualified people to be doing so.
As I said, I do not know what the answer is. The reason the situation is like that is an honest attempt by the agencies to be responsible with taxpayer money; the money is not infinite; they have to decide who to give the money to; and they feel, for reasons that are obvious, that competition is the best way to make sure that the money goes to the best person. I don't think anybody working in science would claim the system is perfect, but other workable solutions are not forthcoming.
There are many examples of competition between in the OSS world. Desktop environments, database engines, programming languages, text editors.
The author of vim told me that at one point he felt pretty content with the program and had not been changing much for a number of years. Then some other guy took on a very active development of another vi clone (forgot the name, sorry) and this gave Bram the incentive to speed up and add many new features to vim. So in this case the competition was beneficial, and I expect that it usually is.
And BTW I don't think competition in science is primarily driven by funding. You need little or no funding for theoretical physics, and yet they're as competitive as anyone or more. The main reason for the very fierce competition in science IMO is that the number of tenured positions is very small compared to the number of PhD graduates. You have to be better than the other guy or else you can kiss your academic career goodbye.
The author of vim told me that at one point he felt pretty content with the program and had not been changing much for a number of years. Then some other guy took on a very active development of another vi clone (forgot the name, sorry) and this gave Bram the incentive to speed up and add many new features to vim. So in this case the competition was beneficial, and I expect that it usually is.
And BTW I don't think competition in science is primarily driven by funding. You need little or no funding for theoretical physics, and yet they're as competitive as anyone or more. The main reason for the very fierce competition in science IMO is that the number of tenured positions is very small compared to the number of PhD graduates. You have to be better than the other guy or else you can kiss your academic career goodbye.
But the reason the number of tenured positions is very small IS because of funding.
Everything comes down to money.
Everything comes down to money.
Could it be enforced, or at least encouraged, by the funding agencies?
That still leaves the problem of reputation-building; a paper where an individual is only one of 20 authors probably isn't so impressive on a CV.
That still leaves the problem of reputation-building; a paper where an individual is only one of 20 authors probably isn't so impressive on a CV.
Not just reputation building, but getting the researcher merit badge in the first place - the PhD. A PhD dissertation is expected to be the work of the student; it's supposed to prove that the student can do self-directed, novel research in their field.
Of course, people do defend dissertations based on work that was a collaboration with others, but I think we're considering a level of collaboration not usually seen.
Of course, people do defend dissertations based on work that was a collaboration with others, but I think we're considering a level of collaboration not usually seen.
That still leaves the problem of reputation-building; a paper where an individual is only one of 20 authors probably isn't so impressive on a CV.
Is the effort expended for the paper able to be divided cleanly into chunks 1/20th the size, to be credited to each author in turn?
Doing research, or any creative work, is not a manufacturing process; the output does not inevitably scale linearly based on the number of inputs. It is possible that a paper could describe something that was very impressive for 20 people to accomplish.
Is the effort expended for the paper able to be divided cleanly into chunks 1/20th the size, to be credited to each author in turn?
Doing research, or any creative work, is not a manufacturing process; the output does not inevitably scale linearly based on the number of inputs. It is possible that a paper could describe something that was very impressive for 20 people to accomplish.
The papers I've seen that have more than a dozen or so authors often work like this: The first and second authors were the project drivers, and did the writeup; the names in the middle were involved in data collection, number-cruching or sub-projects, and are often distributed across different organizations; and the last author (or two) will be a very well-known name, someone who has basically blessed the project by granting their name to it as an established researcher in the area, but who most likely only advised the main researchers.
In general it only counts on your CV if you're the first, second, or maybe third author. But a project with 20 people on it would probably collapse under the organizational weight before completion if it wasn't a pretty big deal. For example, the papers on the Human Genome Project and shotgun sequencing had a lot of authors, with the breakdown as described above.
In general it only counts on your CV if you're the first, second, or maybe third author. But a project with 20 people on it would probably collapse under the organizational weight before completion if it wasn't a pretty big deal. For example, the papers on the Human Genome Project and shotgun sequencing had a lot of authors, with the breakdown as described above.
In the old times, young faculty members who collaborated rather than did original work didn't get tenure and so left "in failure". Graduate students can't collaborate as that's not the purpose of the exercise (as others here have written). Post-docs CAN collaborate, but they're trying to make a name for themselves which is somewhat anti-collaboration.
This leaves the tenured profs, but they're off chasing money to fund their groups. Not a great system, which is why I left it.
This leaves the tenured profs, but they're off chasing money to fund their groups. Not a great system, which is why I left it.
I'd refine that second statement: it's not usually in a graduate student's best interest to collaborate on a project, but the student's PI typically has a competing interest (and a longer time horizon), and so collaborations still happen. The smart PI just manages the risk by assigning collaborative projects to his students -- he gets to reap the rewards of the rare good collaborations, whereas the bad collaborations merely crush the unfortunate grad student to whom they are assigned...and nothing of value is lost.
Part of the game of grad school is recognizing and fending off the "collaborators" who will do nothing but slow you down and/or take credit for your work, while still remaining open to that rare collaboration that can make your career -- the two most successful graduate students I know basically walked into career-defining projects as a result of collaborations that were handed to them in their first year.
Of course, you can be torpedoed by a bad collaboration at any stage of your career, so this isn't a dilemma unique to graduate students. Collaborations are useful only so long as you're getting more than you're putting in.
Part of the game of grad school is recognizing and fending off the "collaborators" who will do nothing but slow you down and/or take credit for your work, while still remaining open to that rare collaboration that can make your career -- the two most successful graduate students I know basically walked into career-defining projects as a result of collaborations that were handed to them in their first year.
Of course, you can be torpedoed by a bad collaboration at any stage of your career, so this isn't a dilemma unique to graduate students. Collaborations are useful only so long as you're getting more than you're putting in.
Academia tries to give you some freedom as if you were managing your own company. But unlike a standard company, you seek funding by submitting proposals to your competition (!).
And unlike a standard company, you have grad students instead of employees who may do what you want and then again may just ignore you.
To top it off, you have to teach as well.
Given all this, do you really think scientists would be open to have whatever remaining freedom they have taken away through greater collaboration?
And unlike a standard company, you have grad students instead of employees who may do what you want and then again may just ignore you.
To top it off, you have to teach as well.
Given all this, do you really think scientists would be open to have whatever remaining freedom they have taken away through greater collaboration?
The empirical finding here is that scientists can sometimes be persuaded to cooperate. How general that could be across how many disciplines, and whether the interesting institutional constraints you mention could be modified to increase cooperation, remains to be seen.
The author impresses upon us that the scientific community is competitive. And that collaboration is rare. These are contradictory ideas.
In competitive environments, collaboration is far from unusual.
In competive environments, it is usual for collaboration to break out between competing individuals. This is especially true in the most cut-throat competitive environments.
In a truly competitive environment, survival depends on the individual gaining a competitive edge. The most effective way for the individual to capture a gain is by joining forces with competitors for mutual gain.
In a truly competitive environment its: collaborate or perish.
In conclusion, the scientific community is not nearly as competitive as the author is inclined to believe.
In competitive environments, collaboration is far from unusual.
In competive environments, it is usual for collaboration to break out between competing individuals. This is especially true in the most cut-throat competitive environments.
In a truly competitive environment, survival depends on the individual gaining a competitive edge. The most effective way for the individual to capture a gain is by joining forces with competitors for mutual gain.
In a truly competitive environment its: collaborate or perish.
In conclusion, the scientific community is not nearly as competitive as the author is inclined to believe.
While I have nothing against collaboration, there is a place for "competition" if you by competition mean "independently confirmed results".
In the specific case I'm most familiar with, there are several research groups making software that does the same thing (n-body simulations, hydrodynamics, radiation transfer) and it is a massive duplicated effort. However, I don't think it's necessarily wasted. In my mind, having everyone collaborate on one code would vastly increase the risk of results being incorrect due to bugs or simply unjustified approximations. When several independent groups perform the same calculation, with different pieces of software using different approximation methods, you can have more faith in that the result is correct.
It's the fact that the academic "system" values being the first to publish a result that pushes everyone towards a competition at all costs approach, but that's only one of the things about incentives for scientists that can be criticized.
I spent my PhD thesis writing a piece of software that I then released under the GPL. It's possible I could get more papers by close-sourcing it and only working with people that would collaborate with me, but I made the calculated gamble that giving the community a free tool in the end would boost my recognition faster due to a wider adoption and more citations.
In the specific case I'm most familiar with, there are several research groups making software that does the same thing (n-body simulations, hydrodynamics, radiation transfer) and it is a massive duplicated effort. However, I don't think it's necessarily wasted. In my mind, having everyone collaborate on one code would vastly increase the risk of results being incorrect due to bugs or simply unjustified approximations. When several independent groups perform the same calculation, with different pieces of software using different approximation methods, you can have more faith in that the result is correct.
It's the fact that the academic "system" values being the first to publish a result that pushes everyone towards a competition at all costs approach, but that's only one of the things about incentives for scientists that can be criticized.
I spent my PhD thesis writing a piece of software that I then released under the GPL. It's possible I could get more papers by close-sourcing it and only working with people that would collaborate with me, but I made the calculated gamble that giving the community a free tool in the end would boost my recognition faster due to a wider adoption and more citations.
Perhaps we just need to put strings on public funding so that study/experiment data is made publicly available (after a short period of exclusivity).
It wouldn't be too different from the Patent system. In that, in exchange for a limited-time monopoly, you agree to publish the details of your invention to further the state of the art.
It might also help scientists learn from each others mistakes, as it would also catalog -failed- projects. Something journals don't do and which doubtlessly results in redundant efforts.
It wouldn't be too different from the Patent system. In that, in exchange for a limited-time monopoly, you agree to publish the details of your invention to further the state of the art.
It might also help scientists learn from each others mistakes, as it would also catalog -failed- projects. Something journals don't do and which doubtlessly results in redundant efforts.
It sounds to me like they need to adopt an open source model of operation. That may not be possible with the current funding system so someone would need to figure out a better way to distribute funding.
Wouldn't it be better for them to be sharing information and results with each other? They would be able to catch mistakes sooner (ie. before publication) and build on and verify each other's work more quickly.
Imagine what it would have been like if the 'cold fusion' fiasco had ben share between more scientists sooner?
Wouldn't it be better for them to be sharing information and results with each other? They would be able to catch mistakes sooner (ie. before publication) and build on and verify each other's work more quickly.
Imagine what it would have been like if the 'cold fusion' fiasco had ben share between more scientists sooner?
Actually, the article is mislabeled, I think. It seems to not so much argue against competition, but rather for the public dissemination of data. Writing papers but withholding the data so that no-one can reproduce your results is very different from simply trying to be the first to complete a study but then publish all data so it's available to others.
In my mind, writing papers but withholding the data that your results are based on is a highly dubious practice.
In my mind, writing papers but withholding the data that your results are based on is a highly dubious practice.
for a scientist, he doesn't make a very scientific argument:
>His experiment in cooperation with four other laboratories, he said, yielded “a very compelling body of data validated by many labs,” and has inspired the researchers to go on freely sharing with one another.”<
Would competition have produced a _more compelling body of data? Very likely if you ask me.
>His experiment in cooperation with four other laboratories, he said, yielded “a very compelling body of data validated by many labs,” and has inspired the researchers to go on freely sharing with one another.”<
Would competition have produced a _more compelling body of data? Very likely if you ask me.
I doubt it. Competing research teams involves a lot of duplicated effort. Novelty is usually required for work to be published, and if you're scooped, there's two options: drop the project, or find some way to differentiate yourself from prior work. In theory, the difference you come up with should make your publication still a worthwhile addition, but in practice, I don't think it usually is.
> Competing research teams involves a lot of duplicated effort.
Competing anything involves massive duplication of effort and wasted talent. One day people will realize this.
http://sciencecareers.sciencemag.org/career_magazine/previou...
"Tournament markets... discard huge amounts of very high-quality talent, training, and skill. In the mythic version, science rewards effort and ability... In the real world, casting off large numbers of extremely capable people is simply how a tournament market works."
For as long as everyone believes that the only alternative to cutthroat competition is sclerotic dictatorship, much human effort will continue to be thrown away in this manner. I give you Be, its patents bought up by patent trolls to prevent its technologies from ever being used:
http://en.wikipedia.org/wiki/BeOS
I give you Symbolics Genera, languishing in obscurity forever because of our society's obsession with property rights.
Competing anything involves massive duplication of effort and wasted talent. One day people will realize this.
http://sciencecareers.sciencemag.org/career_magazine/previou...
"Tournament markets... discard huge amounts of very high-quality talent, training, and skill. In the mythic version, science rewards effort and ability... In the real world, casting off large numbers of extremely capable people is simply how a tournament market works."
For as long as everyone believes that the only alternative to cutthroat competition is sclerotic dictatorship, much human effort will continue to be thrown away in this manner. I give you Be, its patents bought up by patent trolls to prevent its technologies from ever being used:
http://en.wikipedia.org/wiki/BeOS
I give you Symbolics Genera, languishing in obscurity forever because of our society's obsession with property rights.
Would competition have produced a _more compelling body of data?
Possibly not, depending on how the experimental stratification was allocated among the labs. The press release about the research
http://newsroom.ucr.edu/news_item.html?action=page&id=20...
linked to from the submitted article doesn't make clear the nature of the collaboration among the various laboratories, but makes the claim that the previous competitive atmosphere among labs led to results that could not be replicated. Apparently the currently announced results have already been replicated.
Possibly not, depending on how the experimental stratification was allocated among the labs. The press release about the research
http://newsroom.ucr.edu/news_item.html?action=page&id=20...
linked to from the submitted article doesn't make clear the nature of the collaboration among the various laboratories, but makes the claim that the previous competitive atmosphere among labs led to results that could not be replicated. Apparently the currently announced results have already been replicated.
Not sure I agree with this approach. I would rather have the waste of competition than 20 mediocre scientists coming together and putting out dubious research simply to protect reputations. Failure of an individual is much easier to stomach and forget.
I would rather have the waste of competition than 20 mediocre scientists coming together and putting out dubious research simply to protect reputations.
I would rather have neither waste nor dubious research. In the case reported in the submitted article, there is (considering the journal that has agreed to publish the research) an important research finding that had not been achieved under the former system of dueling research labs. In this case, it appears that getting the various labs to collaborate both reduced wastage of research resources and improved the quality of the research result. I have no idea how generalizable this happy result might be in other fields of science.
I would rather have neither waste nor dubious research. In the case reported in the submitted article, there is (considering the journal that has agreed to publish the research) an important research finding that had not been achieved under the former system of dueling research labs. In this case, it appears that getting the various labs to collaborate both reduced wastage of research resources and improved the quality of the research result. I have no idea how generalizable this happy result might be in other fields of science.
Doesn't science already work like a gift economy, at least in parts?
If they didn't compete there wouldn't be any evolution.
[deleted]
What forces them to compete is the money. When your entire research program is based on your having to convince your funding agency to give the bucks to you and not to the other guy, you create competition and moreover you are rewarding the people who are the most competitive.
I agree with the point of the article that sometimes better science would get done with more openness and co-operation (I have seen numerous examples of that in my field), but absent an infinite supply of money I don't see how it can be done. I'd love to know though!