In their 1970 Annual review chapter on Memory and Verbal learning, Tulving and madigan reported that they had independently rated 540 published articles in terms of their "contribution to knowledge". With "remarkable agreement", they found that they had sorted two-thirds of the articles into a category labelled "utterly inconsequential"."The primary function these papers serve is to give something to do to people who count papers instead of reading them. Future research and understanding of verbal learning and memory would not be affected at all if none of the papers in this category seen the light of day"
About 25 percent of the articles were classified as: "run-of-the-mill .. these articles also do not add anything really new to knowledge... [such articles] make one wish that at least some writers, faced with the decision of whether to publish or perish, should have seriously considered the latter alternative"
Only about 10% of the entire set of published papers received the modest compliment of being classified as "worthwhile".Given that memory and verbal learning was then a popular and relatively 'hard' area of psychological research, attracting some of the brightest students, this is a devastating assessment of the end product.
Hence, of the research ideas generated by these psychologists, who are all card-carrying scientists and who liked these ideas well enough to invest weeks or months of their lives working on them, less than 25% of 40% of 10% = 1% actually appear to make some sort of contribution to the discipline.
Given that this work was published in the 1970s when the "publish or perish" pressure was less strongly developed I expect today's number to be worse. Concerning Information Retrieval the field of research of my PhD I would certainly attest that this applies.
In my personal experience the main problem with scientific research today is caused by publication counts representing the sole metric for academic success.
Research is 'optimized' for publications and segmented according to conference schedules. That in itself I consider detrimental to the research effort. However the major damage in terms of advancing resarch in my opinion is caused by the induced tendency of scholars to closely stick to the main stream paradigms, trends and topics. Naturally it is much 'harder' to do research that does not directly build up on the current state of the art. Such research usually takes much longer and therefore results in a lower publication frequency. Secondly it is carrying a much higher risk of not bringing about positive results. Thirdly I deem the chances for acceptance in peer reviewed conferences and journals to be much lower. Such venues are often biased towards current main stream research trends.In my observation there is also an aversion against research that questions the state of the art. People who have spent years or decades to master every aspect of the state of the art have a strong incentive to shoot down anything potentially disruptive.
As such the incentives are misaligned with regard to the aim of science of figuring out new things.
In their 1970 Annual review chapter on Memory and Verbal learning, Tulving and madigan reported that they had independently rated 540 published articles in terms of their "contribution to knowledge". With "remarkable agreement", they found that they had sorted two-thirds of the articles into a category labelled "utterly inconsequential". "The primary function these papers serve is to give something to do to people who count papers instead of reading them. Future research and understanding of verbal learning and memory would not be affected at all if none of the papers in this category seen the light of day"
About 25 percent of the articles were classified as: "run-of-the-mill .. these articles also do not add anything really new to knowledge... [such articles] make one wish that at least some writers, faced with the decision of whether to publish or perish, should have seriously considered the latter alternative"
Only about 10% of the entire set of published papers received the modest compliment of being classified as "worthwhile". Given that memory and verbal learning was then a popular and relatively 'hard' area of psychological research, attracting some of the brightest students, this is a devastating assessment of the end product. Hence, of the research ideas generated by these psychologists, who are all card-carrying scientists and who liked these ideas well enough to invest weeks or months of their lives working on them, less than 25% of 40% of 10% = 1% actually appear to make some sort of contribution to the discipline.
Given that this work was published in the 1970s when the "publish or perish" pressure was less strongly developed I expect today's number to be worse. Concerning Information Retrieval the field of research of my PhD I would certainly attest that this applies.