Scholarly Roulette: Impact Factor & Scientific Quality

A friend and colleague referred me to a recent publication that represents a major indictment of the peer-reviewed journal cartel.

This article summarizes a blog published by what is arguably a Cureus competitor, the open access-publishing house, Frontiers, which in this study could find no correlation between rates of rejection and journal impact factor.

Just for argument’s sake, let’s assume that impact factor does correlate with scientific quality. So if this study is correct, it means by extension that the rate of rejection also doesn’t correlate with quality. In turn this means that the standard peer review process (which totally dominates the journal industry) has no bearing on article quality. In other words, getting published, or not, appears to be a random event.

Accepting this fact (parenthetically one which I already subscribe to) means that despite all the process and rituals of scientific journal publishing, and the huge importance of this matter to society and academic tenure decisions, there is currently no objective index for assessing research article quality. Of course if one believes that impact factor is not a measure of quality, then the above random publication process is even more random than I implied in my argument! Based on this observation, I would like to sarcastically suggest we should rename this current process “scholarly roulette.”

Clearly there are huge shortcomings to the Frontiers study, such as selection bias and the non-medical nature of many scholarly journals. However, to a huge research community that has invested so much of their lives into the existing peer review process these type of findings must be dispiriting. Nevertheless, this current set of observations does resonate with our philosophy at Cureus, where we subscribe to the belief that as long as the medical science within is credible, there is no reason not to publish an article. In other words, we believe that we are, on average, no better equipped to discern quality than any other journal.

A big part of our confidence stems from Cureus’ unique post-publication SIQ scoring system. Although we are no smarter in finding scientific quality up front, we do believe that it is quite possible for the medical community-at-large to find quality over time, post publication. Nevertheless, to make this a reality, every reader has a role to play, which of course includes you!

Does Peer Review = Better Science?

It is widely assumed that the peer review process improves the article that is eventually published. This concept is taken largely as an article of faith except for one miniscule question posed to a range of published international academics as part of a large survey in 2009; in this survey, 91% of respondents, none of whom were medical professionals, agreed with the idea that the revisions made to their article as a result of the conventional peer review process improved the end product. Of note, most of this subjective sense of manuscript improvement was felt by authors to be within the discussion section. The response to this one question represents the only evidence of which I am aware that demonstrates peer review resulting in better published papers.

Obviously one cannot simply dismiss the findings in the above study out of hand. However, it seems to be worth asking whether or not peer review has the same advantages outlined in the above survey when considered in the rich world of observational studies that describes so much of medical publishing. Moreover, is an improvement in a article’s discussion section significant enough to warrant all the added time and effort on the part of the very busy physician authors who write most medical articles and who represent some of the busiest individuals on our planet?

As an editor of a journal myself (Cureus) I cannot help but ask these questions. I also pose this line of inquiry now, however, due to a recent discussion with a colleague who is a very senior and internationally respected neurosurgeon. He is about to publish an article in a well-known neurosurgery journal having now jumped through the usual reviewer hoops requiring several revisions and months of effort. I asked this colleague whether his article was any better after all this effort. His simple response: “NO!” In fact, in my own academic career, which involves more than 150 peer-reviewed articles, I find it difficult to recall a single instance where I felt the final published article was meaningfully improved through the peer review process.

Oh, I jumped through plenty of hoops, like routinely needing to add additional references so that my paper would cite the reviewer’s own publications, but sadly I have found far too much of the peer review process to be merely an exercise in power and vanity. With so many diseases so poorly treated, and so much clinical research that needs to (or could) be published, is all this inefficiency justifiable? It is for this very reason that we at Cureus emphasize a much more efficient post-publication “peer review” process, or as we term it, SIQ (Scholarly Impact Quotient). I would very much like to hear from any readers of this blog who have published in conventional peer-reviewed journals  – did the review  process improve the ultimate published article?

Scientific Publishing: No Need To Suffer

Scientific publishing is a long established process, often presenting its share of frustrations to authors. Lengthy peer review processes and subsequent rejections can prevent timely publication of important research. A recent article in the Economist (June, 2013) addressed the issue of scientific publishing, stating that “slowness” is often identified by authors as the primary drawback of the publication process. According to the article, the average time for publication is six months after submission, and frequently follows submissions to multiple journals, as the article may be rejected for publication by journal after journal despite high scientific quality. This contributes to the extended time to publication. In addition, the article must be peer reviewed by multiple experts in the field, which adds  to the time for publication. This has been particularly true for print journals, in which the article, once approved, must also wait for publication of the particular edition of the journal.

A new generation of journals has been created in recent years. Cureus.com is an online medical journal with an Editorial Board that consists of hundreds of experts in a wide variety of medical specialties. The goal of the Cureus journal is to provide a transparent and rapid publishing process in which authors receive feedback, and an unbiased measure of the scientific merit of their papers via a novel crowd-sourcing algorithm, the SIQ, from reviewers and all readers. Authors on Cureus have the ability to have their work published in a timely manner, for no cost, and while retaining copyright at all times. The online accessibility also enables patients to access the most recent research on their particular medical condition. The structure and publication process provided by Cureus addresses and eliminates the historical problems in scientific publishing. Most importantly, it allows important research to be published quickly, and to be viewed by medical professionals and patients alike.

Time is Critical in Recovery from Stroke

hourglassStroke is a leading cause of death in the United States, killing one American every 4 minutes, according to the Centers for Disease Control and Prevention. A stroke is an interruption of the blow flow to the brain. Strokes and be caused by a blocked blood vessel, or one that bursts and bleeds into the brain. Strokes can occur at any point in life, but the risk of stroke increases with age. Each year, over 795,000 Americans have a stroke, resulting in long-term or permanent disability.

The speed at which a stroke victim receives treatment can mean the difference between recovery and long term, or even fatal, consequences.  This was highlighted in a recent study published in the Journal of the American Medical Association by Dr. Jeffrey Saver, director of UCLA’s Comprehensive Stroke Center and his colleagues. In their study, Dr. Saver et al., found that the risk of suffering from stroke symptoms or death decreases by 4% for each 15 minutes doctors gain when intervening with a stroke victim. John Adler, MD, a Dorothy & TK Chan Professor of Neurosurgery and Radiation Oncology at Stanford University, and CEO of Cureus, Inc., stated “This paper by Saver et al. adds to a large body of now incontrovertible data.  In the setting of ischemic stroke, the speedy restoration of brain perfusion with tPA leads to better functional outcomes.  The big unmet public health challenge is ensure the fastest possible recognition of ischemic stroke to be followed by ever more timely therapeutic intervention.”

Common signs of stroke can include sudden numbness or weakness of the face, arm or leg, confusion, trouble seeing, loss of balance or coordination, or a sudden, severe headache. At risk are those with diabetes, poor diet, obesity, physical inactivity, and those who consume excessive amounts of alcohol. Concerned about your own risk of stroke? Use this stroke calculator developed at the UCLA Stroke Center: