It is widely assumed that the peer review process improves the article that is eventually published. This concept is taken largely as an article of faith except for one miniscule question posed to a range of published international academics as part of a large survey in 2009; in this survey, 91% of respondents, none of whom were medical professionals, agreed with the idea that the revisions made to their article as a result of the conventional peer review process improved the end product. Of note, most of this subjective sense of manuscript improvement was felt by authors to be within the discussion section. The response to this one question represents the only evidence of which I am aware that demonstrates peer review resulting in better published papers.
Obviously one cannot simply dismiss the findings in the above study out of hand. However, it seems to be worth asking whether or not peer review has the same advantages outlined in the above survey when considered in the rich world of observational studies that describes so much of medical publishing. Moreover, is an improvement in a article’s discussion section significant enough to warrant all the added time and effort on the part of the very busy physician authors who write most medical articles and who represent some of the busiest individuals on our planet?
As an editor of a journal myself (Cureus) I cannot help but ask these questions. I also pose this line of inquiry now, however, due to a recent discussion with a colleague who is a very senior and internationally respected neurosurgeon. He is about to publish an article in a well-known neurosurgery journal having now jumped through the usual reviewer hoops requiring several revisions and months of effort. I asked this colleague whether his article was any better after all this effort. His simple response: “NO!” In fact, in my own academic career, which involves more than 150 peer-reviewed articles, I find it difficult to recall a single instance where I felt the final published article was meaningfully improved through the peer review process.
Oh, I jumped through plenty of hoops, like routinely needing to add additional references so that my paper would cite the reviewer’s own publications, but sadly I have found far too much of the peer review process to be merely an exercise in power and vanity. With so many diseases so poorly treated, and so much clinical research that needs to (or could) be published, is all this inefficiency justifiable? It is for this very reason that we at Cureus emphasize a much more efficient post-publication “peer review” process, or as we term it, SIQ (Scholarly Impact Quotient). I would very much like to hear from any readers of this blog who have published in conventional peer-reviewed journals – did the review process improve the ultimate published article?