Academic journals face a radical shake-up(economist.com)
economist.com
Academic journals face a radical shake-up
http://www.economist.com/node/21559317
6 comments
Hi, I'm head of technology for eLife, the journal mentioned in economist article. We have a remit from Wellcome, and the other funders, to build products that go beyond academic journals. I'll be putting together a small team over the next few months, based in Cambridge UK, to work on ideas. It will most be a factory for prototyping ideas about harnessing attention to researchers work. If you might be interested drop me a line @ianmulvany on twitter, or [email protected].
Like Ian and eLife, I am putting together a tech team for PeerJ [1], a new Open Access publisher with venture backing from Tim O'Reilly [2,3]. If you are in London and want to work on a problem that truly affects all of us then give me shout at [email protected]. Looking for both dev ops backend and front-end/ux.
1. http://peerj.com/careers/ 2. http://pandodaily.com/2012/06/12/peerj-raises-950k-from-tim-... 3. http://www.nature.com/news/journal-offers-flat-fee-for-all-y...
1. http://peerj.com/careers/ 2. http://pandodaily.com/2012/06/12/peerj-raises-950k-from-tim-... 3. http://www.nature.com/news/journal-offers-flat-fee-for-all-y...
> Here, manuscripts are subject to a ruthless process of open peer review, rather than the secret sort traditional publishers employ.
I don't know what this means. If the arXiv has some sort of open but centralized peer review, I haven't seen it. Maybe the economist just means that, once posted on the arXiv, a paper can be reviewed by anyone who wants and they can post on their blog. But this would be true of self-published papers, and no one would call that meaningful peer review.
> An arXived paper may end up in a traditional journal, but that is merely to provide an imprimatur for the research team who wrote it. Its actual publication, and its value to other scientists, dates from its original arrival online.
This isn't quite true. Depending on how lazy the journal referees are feeling, they can provide valuable feedback. Papers can change substantially during the review process (although this doesn't happen more often than it does). Typically, physicists update the arXiv version to reflect these changes.
I don't know what this means. If the arXiv has some sort of open but centralized peer review, I haven't seen it. Maybe the economist just means that, once posted on the arXiv, a paper can be reviewed by anyone who wants and they can post on their blog. But this would be true of self-published papers, and no one would call that meaningful peer review.
> An arXived paper may end up in a traditional journal, but that is merely to provide an imprimatur for the research team who wrote it. Its actual publication, and its value to other scientists, dates from its original arrival online.
This isn't quite true. Depending on how lazy the journal referees are feeling, they can provide valuable feedback. Papers can change substantially during the review process (although this doesn't happen more often than it does). Typically, physicists update the arXiv version to reflect these changes.
The article makes a bit of an error. Publishers do provide a service. They organise peer review, in which papers are criticised anonymously by experts (though those experts, like the authors of papers, are rarely paid for what they do).
It's actually the minimally paid editors who do this. All the journal does is provide a crappy web app to organize the process.
It's actually the minimally paid editors who do this. All the journal does is provide a crappy web app to organize the process.
I'm not a publisher but have worked at publishers in the past.
> All the journal does is provide a crappy web app > to organize the process.
I'm guessing you're being facetious ;) but that's not true. It depends on the publisher but they may also edit the manuscript; find the right reviewers; have a full time editor make a decision based on the reviewer's feedback; typeset or convert it to machine readable format (authors submit in Word, but this'll often be converted to an industry standard XML format to allow interoperability between different databases - both abstract databases and publisher ones as journals change publishers frequently); ensure the content is properly stored (this is not the same as putting it up on S3 for 2 cents a month. It's the scholarly record and publishers generally take the responsibility of maintaining it seriously. Publishers pay a fee to put content into a darknet run by a consortia of libraries - if the publisher goes out of business the consortia makes all the content available for free. They also pay membership fees to industry organizations like CrossRef that allow them to deposit canonical metadata in a central database and mint DOIs) and discoverable (at the end of the day your paper will get much more attention when published in Science than it does your blog).
Are all of those things worth paying more than $99 an article for? It's debatable.
But can anybody with a Wordpress blog do what publishers can? No.
(Same sort of thing applies to trade. Look at self-published books. You can create great content for free and stick it in an EPUB but you'll rapidly find that you need to start shelling out to be included in the book databases bricks and mortar stores use, to have a cover designed, to get an ISBN, to start actually selling through Amazon or wherever... publishing isn't as trivial as it seems.)
> All the journal does is provide a crappy web app > to organize the process.
I'm guessing you're being facetious ;) but that's not true. It depends on the publisher but they may also edit the manuscript; find the right reviewers; have a full time editor make a decision based on the reviewer's feedback; typeset or convert it to machine readable format (authors submit in Word, but this'll often be converted to an industry standard XML format to allow interoperability between different databases - both abstract databases and publisher ones as journals change publishers frequently); ensure the content is properly stored (this is not the same as putting it up on S3 for 2 cents a month. It's the scholarly record and publishers generally take the responsibility of maintaining it seriously. Publishers pay a fee to put content into a darknet run by a consortia of libraries - if the publisher goes out of business the consortia makes all the content available for free. They also pay membership fees to industry organizations like CrossRef that allow them to deposit canonical metadata in a central database and mint DOIs) and discoverable (at the end of the day your paper will get much more attention when published in Science than it does your blog).
Are all of those things worth paying more than $99 an article for? It's debatable.
But can anybody with a Wordpress blog do what publishers can? No.
(Same sort of thing applies to trade. Look at self-published books. You can create great content for free and stick it in an EPUB but you'll rapidly find that you need to start shelling out to be included in the book databases bricks and mortar stores use, to have a cover designed, to get an ISBN, to start actually selling through Amazon or wherever... publishing isn't as trivial as it seems.)
I'm only familiar with math publishing, but I've never heard of a math journal that did anything other than provide a web form for the editor to request a review (and similarly, for the reviewer to submit with).
Typesetting is usually done (correctly) by latex, and then possibly screwed up by some human the publisher outsourced to. Automated typesetting is a solved problem, and there is no need for humans.
As for your cost of $99/article, the Arxiv costs $400k/year (though admittedly, Donald Knuth does the typesetting).
As for your "darknet", are you really going to claim this darknet is more reliable than mirroring articles to 3-4 amazon regions plus rackspace files? I'm sure the darknet costs at least 10x as much, but that isn't the issue. Similarly, paying membership fees to the other big publishers to keep little guys out of the market is again not the issue.
If it weren't for reputation effects and agency costs (i.e., if the market were competitive), this process would cost well under $99/article. The only reason the publishers haven't already died is because tenure committees want to see Journal of Computational Physics rather than the Upstart Computational Physics Journal.
Typesetting is usually done (correctly) by latex, and then possibly screwed up by some human the publisher outsourced to. Automated typesetting is a solved problem, and there is no need for humans.
As for your cost of $99/article, the Arxiv costs $400k/year (though admittedly, Donald Knuth does the typesetting).
As for your "darknet", are you really going to claim this darknet is more reliable than mirroring articles to 3-4 amazon regions plus rackspace files? I'm sure the darknet costs at least 10x as much, but that isn't the issue. Similarly, paying membership fees to the other big publishers to keep little guys out of the market is again not the issue.
If it weren't for reputation effects and agency costs (i.e., if the market were competitive), this process would cost well under $99/article. The only reason the publishers haven't already died is because tenure committees want to see Journal of Computational Physics rather than the Upstart Computational Physics Journal.
Don't get me wrong - I don't think that the existing system works particularly well or is value for money, I just wanted to point out that there's much more going on than hosting a crappy website. ;)
> only familiar with math publishing
Fair enough. I'm only really familiar with the life sciences.
> Automated typesetting is a solved problem
This isn't as simple outside of CS/EE/maths unfortunately. Most academic researchers couldn't tell you what LaTeX is.
> As for your cost of $99/article, the Arxiv costs $400k/year
arXiv is awesome but it's a preprint repository not a journal (whether or not a repo + engaged community is as 'good' as a peer reviewed journal is probably a whole other thread... physicists still usually publish in journals, even though they use arXiv constantly and put their manuscripts up there first)
I picked $99 because it's the number PeerJ (also awesome) is suggesting. In reality life science journals will charge you $1k+...
> are you really going to claim this darknet is more reliable than mirroring articles
I don't make that claim, academic librarians and publishers do.
(if you prefer you can swap out this cost and replace it with the cost of hiring somebody to do ops for you on an occasional, freelance basis to handle things going wrong, moving to new cloud providers etc. over a 25 year period...).
> Similarly, paying membership fees to the other big publishers to keep little guys out of the market
I don't think I explained this the right way. The fees are to help maintain common services that all publishers then use - for example, to ensure that given an identifier you can always find the full text of a paper (URLs being unstable over even relatively short timeframes), to mint and store metadata about those identifiers to make sure the right people get credit when cited etc. Those services are unquestionably valuable and get used by millions of scientists every day.
> If it weren't for reputation effects and agency costs (i.e., if the market were competitive), this process would cost well under $99/article
I disagree but believe that you should be right. People like PeerJ, eLife, arXiv and others need to keep exploring alternative ways of doing things until we hit on a sustainable model.
> only familiar with math publishing
Fair enough. I'm only really familiar with the life sciences.
> Automated typesetting is a solved problem
This isn't as simple outside of CS/EE/maths unfortunately. Most academic researchers couldn't tell you what LaTeX is.
> As for your cost of $99/article, the Arxiv costs $400k/year
arXiv is awesome but it's a preprint repository not a journal (whether or not a repo + engaged community is as 'good' as a peer reviewed journal is probably a whole other thread... physicists still usually publish in journals, even though they use arXiv constantly and put their manuscripts up there first)
I picked $99 because it's the number PeerJ (also awesome) is suggesting. In reality life science journals will charge you $1k+...
> are you really going to claim this darknet is more reliable than mirroring articles
I don't make that claim, academic librarians and publishers do.
(if you prefer you can swap out this cost and replace it with the cost of hiring somebody to do ops for you on an occasional, freelance basis to handle things going wrong, moving to new cloud providers etc. over a 25 year period...).
> Similarly, paying membership fees to the other big publishers to keep little guys out of the market
I don't think I explained this the right way. The fees are to help maintain common services that all publishers then use - for example, to ensure that given an identifier you can always find the full text of a paper (URLs being unstable over even relatively short timeframes), to mint and store metadata about those identifiers to make sure the right people get credit when cited etc. Those services are unquestionably valuable and get used by millions of scientists every day.
> If it weren't for reputation effects and agency costs (i.e., if the market were competitive), this process would cost well under $99/article
I disagree but believe that you should be right. People like PeerJ, eLife, arXiv and others need to keep exploring alternative ways of doing things until we hit on a sustainable model.
> (if you prefer you can swap out this cost and replace it with the cost of hiring somebody to do ops for you on an occasional, freelance basis to handle things going wrong, moving to new cloud providers etc. over a 25 year period...).
Plenty of countries have national/legal deposit libraries (my country has six of them) and the costs of running them are a rounding error in national terms. You can hardly claim an electronic national library would cost more to run than a physical library - much smaller volume of stuff to store, no staff to retrieve and shelve documents, no in person visitors.
Plenty of countries have national/legal deposit libraries (my country has six of them) and the costs of running them are a rounding error in national terms. You can hardly claim an electronic national library would cost more to run than a physical library - much smaller volume of stuff to store, no staff to retrieve and shelve documents, no in person visitors.
I take it you haven't submitted to many CS journals ;) Last time I made a submission to the ACM I had to essentially hand in a publication ready pdf, this is true for the vast majority of CS journals. Additionally I've published in open access journals outside of CS where I did just have to submit a word doc and they took care of the rest.
Additionally I've known plenty of editors who do many of the things you mentioned above for free (for paid journals) with the only reward being bonus points for tenure.
Considering that there the costs to creating software are much higher (simply because unlike journal articles there's no real incentive for devs to give away there rather price labor, other than just wanting to give back) and we still have a thriving OSS eco-system I find it very hard to believe that we can't easy substitute out the very few parts of academic journals that do incur some sort of real cost for the publisher with volunteer work.
Additionally I've known plenty of editors who do many of the things you mentioned above for free (for paid journals) with the only reward being bonus points for tenure.
Considering that there the costs to creating software are much higher (simply because unlike journal articles there's no real incentive for devs to give away there rather price labor, other than just wanting to give back) and we still have a thriving OSS eco-system I find it very hard to believe that we can't easy substitute out the very few parts of academic journals that do incur some sort of real cost for the publisher with volunteer work.
"Publishers pay a fee to put content into a darknet run by a consortia of libraries - if the publisher goes out of business the consortia makes all the content available for free"
So driving them into ruin collectively frees the back catalog? Since access to old papers are a major concern in every plan to cut the middlemen, that would actually be good news!
So driving them into ruin collectively frees the back catalog? Since access to old papers are a major concern in every plan to cut the middlemen, that would actually be good news!
With all respect to PeerJ and eLife, I'm hoping that we can do a bit better than replacing monolithic, closed-access publishers with monolithic open-access publishers. Personally, I'm hoping that something akin to the Wordpress model takes off - supply powerful journal publishing tools free/cheap and let a thousand journals bloom. Scholastica [1] is the only one taking this approach that I'm aware of (disclosure: friends). It started about a year ago by several University of Chicago students and seems to be getting some nice traction, starting with law journals.
[1] https://scholasticahq.com/
[1] https://scholasticahq.com/
I disagree -- an explosion of micro- and self-publishing would make it exponentially harder to stay abreast of new work. I can stay more-or-less up to speed on my field by attending two or three conferences and reading two journals (OK, so it's kind of a small field). If everyone who previously published in those four locations suddenly decided to self-publish I'd have to follow a couple hundred blogs/microjournals. Keeping up with all of those resources would be a non-trivial amount of work, not to mention the drastic drop in peer review quality that would likely result.
Step forward the new Publishers re-purposed as curated search engines.
There is a real possibility that all you would end up with is an on-line version of the current crop of dead-tree publishers. In addition to simply democratising publishing there needs to be some thought applied to the subject of the publisher as an institution. Learned societies and universities fit this approach so tools that made publishing easier for these group to manage would be a step forward. However any concentration of power or influence results in some of the current set of problems quickly resurfacing. Perhaps it is simply a facet of any enterprise where humans are involved and a revolution every so often is needed to shake things up a little.
There is a real possibility that all you would end up with is an on-line version of the current crop of dead-tree publishers. In addition to simply democratising publishing there needs to be some thought applied to the subject of the publisher as an institution. Learned societies and universities fit this approach so tools that made publishing easier for these group to manage would be a step forward. However any concentration of power or influence results in some of the current set of problems quickly resurfacing. Perhaps it is simply a facet of any enterprise where humans are involved and a revolution every so often is needed to shake things up a little.
But publication and aggregation no longer need to be performed together. That was a limitation of print publishing, since the cost of publication was high enough that you really needed to do your aggregation/filtering first.
Now the act of publishing is trivially cheap, so you can simply let everyone publish everything first, and apply aggregation and filtering second.
If you took all the attendees of those two conferences and gave them a collaborative website for rating and reviewing papers, you'd probably get something just as good as the old journals, and probably faster.
Now the act of publishing is trivially cheap, so you can simply let everyone publish everything first, and apply aggregation and filtering second.
If you took all the attendees of those two conferences and gave them a collaborative website for rating and reviewing papers, you'd probably get something just as good as the old journals, and probably faster.
Then you could follow those blogs, pick out the best articles, and link to the good ones. You could even offer constructive critisism (i.e. "I'll link to you if you expand a bit on how you did step 3"). Kind of like a journal.
I could indeed. Only under the current system there's someone else, probably more qualified than me, already doing it for me. That's part of what's so great about peer review -- other people do it for you almost all of the time.
So, this other person ... what motivates them to do it? They don't get paid, and they don't really care about the journals (per say). Presumably, they do it for the good of their profession, or for fame, or because having a lot of influence in the field makes it easier for them to secure tenure / promotions / grant money.
We do it for the good of our profession. All I'm asserting is that the difference between the current system and a self-publishing free-for-all is that under the current system peer review for any single publication is performed by a small (3-5) number of people. I might review one or two publications per year (for no financial compensation, as you say) -- but in exchange for this effort I get to read dozens of papers which have been peer reviewed to a similar standard. In a free-for-all I'd have to put in the effort of peer reviewing every paper I read. That's a huge amount of work.
> In a free-for-all I'd have to put in the effort of peer reviewing every paper I read.
You seem to be criticizing a straw man. Nobody is suggesting that peer review should stop. We're suggesting that there's no longer any reason to link peer review with publication and access.
The real value is added by the reviewers and the authors, and they can keep on providing that value the same as always. It's the publisher who has become nearly superfluous, yet it's the publisher who charges boatloads of money for access.
You seem to be criticizing a straw man. Nobody is suggesting that peer review should stop. We're suggesting that there's no longer any reason to link peer review with publication and access.
The real value is added by the reviewers and the authors, and they can keep on providing that value the same as always. It's the publisher who has become nearly superfluous, yet it's the publisher who charges boatloads of money for access.
I also disagree with a proliferation of micro self publishing without having an aggregation systems in place first. For the HN community it is obvious that the aggregation can be done post-publication and in theory there are many different ways to achieve this. However, it somehow never gets done. There is no Techmeme for science, no real effort to create filtering tools for the stream of scientific articles that are produced daily. Maybe it is a small market but there is a market. I would pay for a service that would give me a (useful) personalized stream of articles. Without the filtering/aggregation the idea of self publishing or the current trend for open access mega journals (PLoS ONE, Scientific Reports, etc) make content discovery a challenge.
I imagine that sites akin Hacker News would arise to help filter and curate each field. They could range from highly professional, well-financed operations to a prominent academic who catalogs interesting papers on her blog to a popularizer who focuses on exposing/explaining journal papers to a lay audience. Academic could use tools like Scholastica so self-publish in dozens of small sites free of charge - but they could also use Scholastica to replicate the 3-4 journals seen today. I don't think it's a given that a field will be harder to track - if anything, discovering great work would seem easier if more voices were able to play a role.
> I'd have to follow a couple hundred blogs/microjournals.
RSS?
RSS?
I in no way disagree with this sentiment, and I am a very big fan of what Scholastica are doing, I had a really great chat to them a few weeks ago.
Scientific publishing seems from the outside to not really be making as much use of the web as a platform, as it should. Paywalls have a big part to play in that, as do contracts with non-disclosure clauses over how much libraries pay to publishers.
Open access and a move to article processing charges can go some way towards moving the needle on that.
One of the many other issues to content with (and there are many), is moving the behaviour of academics, who tend to be quite conservative. I'm hoping that with the backing of three of the biggest research funders in science I can get the attention of researchers who would not normally be early adopters of innovative approaches. I think once that shift starts to happen the door can be opened to a yet more radical re-thinking of the role of the journal.
Scientific publishing seems from the outside to not really be making as much use of the web as a platform, as it should. Paywalls have a big part to play in that, as do contracts with non-disclosure clauses over how much libraries pay to publishers.
Open access and a move to article processing charges can go some way towards moving the needle on that.
One of the many other issues to content with (and there are many), is moving the behaviour of academics, who tend to be quite conservative. I'm hoping that with the backing of three of the biggest research funders in science I can get the attention of researchers who would not normally be early adopters of innovative approaches. I think once that shift starts to happen the door can be opened to a yet more radical re-thinking of the role of the journal.
More open-access publication could enable some really fascinating meta research. There's a really active set of researchers in computational linguistics who study the structure and language of scientific papers, but we tend to focus on AI and similar domains -- partly because those are the most familiar to us, but presumably also due to ease of access (bless the ACM archive...). Meta-research on different dialects of science could be insanely interesting.
...hmm...
...hmm...