Scholarly Roulette: Impact Factor & Scientific Quality

A friend and colleague referred me to a recent publication that represents a major indictment of the peer-reviewed journal cartel.

This article summarizes a blog published by what is arguably a Cureus competitor, the open access-publishing house, Frontiers, which in this study could find no correlation between rates of rejection and journal impact factor.

Just for argument’s sake, let’s assume that impact factor does correlate with scientific quality. So if this study is correct, it means by extension that the rate of rejection also doesn’t correlate with quality. In turn this means that the standard peer review process (which totally dominates the journal industry) has no bearing on article quality. In other words, getting published, or not, appears to be a random event.

Accepting this fact (parenthetically one which I already subscribe to) means that despite all the process and rituals of scientific journal publishing, and the huge importance of this matter to society and academic tenure decisions, there is currently no objective index for assessing research article quality. Of course if one believes that impact factor is not a measure of quality, then the above random publication process is even more random than I implied in my argument! Based on this observation, I would like to sarcastically suggest we should rename this current process “scholarly roulette.”

Clearly there are huge shortcomings to the Frontiers study, such as selection bias and the non-medical nature of many scholarly journals. However, to a huge research community that has invested so much of their lives into the existing peer review process these type of findings must be dispiriting. Nevertheless, this current set of observations does resonate with our philosophy at Cureus, where we subscribe to the belief that as long as the medical science within is credible, there is no reason not to publish an article. In other words, we believe that we are, on average, no better equipped to discern quality than any other journal.

A big part of our confidence stems from Cureus’ unique post-publication SIQ scoring system. Although we are no smarter in finding scientific quality up front, we do believe that it is quite possible for the medical community-at-large to find quality over time, post publication. Nevertheless, to make this a reality, every reader has a role to play, which of course includes you!

Exceptional Responders in Oncology: Winning Case Report Announced!

51 published case reports. More than 1,200 SIQ scorings. Over 12,000 views. Only 1 winner.

cancercommons-email-winner

Congratulations to Mark ZakiPam LaszewskiNatasha RobinetteHusain SalehNaweed RazaAmmar Sukari & Harold Kim! Their case report, entitled ‘Unresectable Extraskeletal Myxoid Chondrosarcoma of the Neck: Early Tumor Response to Chemoradiotherapy,’ received the highest SIQ score, with an 8.2 (which has since risen to a stellar 8.5).

In addition to the winning article, we received dozens of high-quality case reports from around the world, all of which are now published and available free of charge.

We’d like to extend a big thank you to the Cureus community for their efforts in reading and scoring dozens of case reports over the past few months. Without you, this competition would not be possible – you’ve helped shine a spotlight on dozens of interesting cases of exceptional responders in oncology.

And remember – even though the competition is over, you can still access and score any of the 51 case reports. Stay tuned for news about our next competition – launching soon!

The Anecdote is the Antidote for What Ails Modern Medical Science

Today I read with great interest an article in Vox.com entitled: “Fleming’s discovery of penicillin couldn’t get published today. That’s a huge problem.” As a long time academician and Co-Editor-in-Chief of Cureus, I could not agree more! Not only would Fleming’s discovery go unreported, but Jenner’s discovery of the first small pox vaccine or Connecticut dentist Crawford Long’s discovery of ether anesthesia would also be dismissed as trivial curiosities and anecdotes.

In a rapidly changing world of medical science, the process of peer-reviewed publishing has never been more important for validating medical knowledge, yet at the same time the arcane processes of peer-reviewed journals have never been more at odds with potential societal benefits. The altruism that is supposed to drive the publication of scientific research has been almost entirely co-opted by the peculiar needs of academic promotion and tenure, as well as the pecuniary demands of the scholarly publishing industry; the public good of medical knowledge has been reduced to a mere after-thought by both academia and the publishing industry. These two interest groups only seek seemingly important “big science,” and actually decry any role for mere clinical observation, which Cureus has in the past and the Vox authors now in the present both refer to, in a somewhat ironic, self-deprecating manner, as “small science.”

Despite the above dire (my word) situation in medical publishing, just maybe the proverbial pendulum is about to swing the other way. For example, the conventional wisdom is that we are entering an era of “personalized medicine,” a concept which at its core recognizes the unique genomic, proteomic, metabolomics and demographic, etc. profile of every patient. Embodied within this concept is the understanding that each of us will as patients undergo a unique clinical experience, not necessarily predicted by a priori knowledge, and that such observations will need to be documented if we are to build new scientific understandings. Assuming that to be true, the future of medical publishing may well be found in observational “small science.” If this is true, the important medical publishing of the future may be about to shift from “the few” elites reporting big science, to the power of the many, within the trenches of medicine, collectively reporting their humble clinical observations? For the record, Cureus aspires to lead this transition into the new world of small, yet very powerful, medical science.

Intellectual Fascism

I have been Co-Editor-in-Chief of Cureus for about three years and in this time I’ve learned a lot about how peer-reviewed journals function. It’s often said that in water polo the real game happens beneath the surface of the water. Similarly with journals, sometimes the serious action goes on behind the scenes. From its inception, Cureus was designed to minimize the role of politics in scientific publishing by way of its post-publication SIQ scoring process. Despite these ambitions, politics have occasionally crept into our efforts to publish great medical science. So it was with one recent article, and boy did the Cureus editorial staff learn a lot through this experience!

The article in question was written by several very accomplished clinical neuroscientists and involved a complex intersection of multiple scientific fields. Despite being evaluated by three reviewers, a clear error was noted in the published article by a reader; the error was of a political nature and not scientific in the least, but still an unambiguous error. An erratum was being prepared when a big hullabaloo broke loose in social media. Two individuals, whose specialty overlapped the erroneous article, attacked the article for its political misstatement, and by extension, Cureus’ journalistic integrity for missing this error during our pre-publication review process.

I immediately invited these critics to set the record straight via our liberal comment and scoring processes, but in a series of personal (and necessarily confidential) emails, the critics refused, insisting on remaining anonymous. Over the next several days they recruited a chorus of similarly-minded colleagues who insisted that the article in question represented serious scientific misconduct and demanded it be retracted… period! In light of these demands, Cureus, by virtue of its status as a peer-reviewed journal, was obligated to investigate under ICJME (International Committee of Medical Journal Editors) guidelines.

I personally oversaw the investigation, which started by recruiting seven truly world-class domain experts, who after reading both the original and the proposed corrected manuscripts, were to advise me; I deliberately included a couple of researchers suggested by the critics of the article. In parallel, I stumbled upon the existence of a listserv community of likeminded researchers including the two critics, whose major modus operandi is to fiercely act en-mass, hyena-like, oftentimes via social media, when certain partisan political issues arise, such as the article Cureus had unwittingly published.

If ever I witnessed intellectual fascism, this was it; the only thing missing was a goose-stepping mustached man. However, this was also to be a moment of truth for our young journal. Pending the advice of the seven domain experts, would Cureus stand up for open scientific discourse? Or would we join the ranks of cowering researchers?

After almost one month of analysis by the aforementioned unimpeachable panel of experts, some of whom are at the very pinnacle of their respective fields, it was determined that the Cureus article had erred badly (yet seemingly inadvertently) in misstating a political reality. However, the science itself was credible. As a result, an erratum addressing the erroneous facts was published and the original article retracted.

Ultimately this experience reminds me, and by virtue of this blog should remind all readers, that standing up for open and honest scientific discussion, devoid as much as possible from political considerations, is a constant struggle even in our supposedly democratic world. In fact I invite the very critics of the article in question to now publish their own scientific concepts with Cureus, which perhaps might even refute the published paper. The function of journals is not to anoint “scientific truths,” but to provide a forum for scientific truths to be discovered, and refuted.

We at Cureus are especially fortunate to have such a liberal post-publication comment and scoring system. This process, which is available to everyone, is intended to provide a voice for even the most contrarian scientific ideas. The power of the Cureus community-at-large remains a great potential strength. Please don’t be afraid. Step up and use it!

In Response to Allegations Concerning Internet Medical Society

Allegations were recently leveled against iMed.pub, an organization affiliated with one of our channel partners, Internet Medical Society. We take such matters very seriously. We have conducted a thorough investigation and we’re here to share our thoughts on this situation as we move forward.

After considering the allegations and speaking at length with the IMS channel administrator, Manuel Menendez, we can state that we believe in the sincerity of IMS’s ambition to publish credible medical scientific articles. While the accusations are certainly nothing to take lightly, we feel confident that work being submitted to Cureus via the IMS channel is of suitable quality and repute. Furthermore, via our wholly unique post-publication review and scoring system, Cureus is in a unique position to uncover fraudulent and illegitimate science, as we actively encourage our community to examine and review articles long after they’ve passed through peer review.

It is worth reminding all readers of scientific publications to always interpret the content with a skeptical eye, whether the article is published in Nature, The New England Journal of Medicine or Cureus. Healthy skepticism is always at the heart of science. It is only through the lens of time that scientific ideas can be truly validated, which is why we at Cureus advocate for our process of post-publication review via SIQ.

As a result, Cureus will continue to host the Internet Medical Society channel and looks forward to the channel’s continued success. As has been the case since our launch in 2011, Cureus will not tolerate plagiarism or illegitimate or untoward publication practices and will remain ever-watchful for any signs of potential transgressions.

Banished From Cureus: Introducing a New Cureus Editorial Policy

Founded with the belief that far too many credible physician and allied medical specialists are disenfranchised by a publishing system driven largely by money and academic promotion, Cureus has, from its inception, bent over backwards to remove barriers to medical publication. Whenever possible, we have always tried to give each author the benefit of doubt throughout the entire review and publishing cycle. In the process of being so liberally minded, however, Cureus has attracted a handful of prospective authors that seek to take advantage of our generosity. In particular this group of authors has either failed to read closely Cureus’ author instructions or chosen to not follow its unambiguous dictates. This is especially apparent in the areas of copy editing and formatting, for even the simplest little requirements, such as bracketing reference citations.

Why is this handful of authors so sloppy? Maybe having been schooled in the practices of other journals they assume some faceless (yet compensated) copy editing team will clean up their carelessness? Of course other journals will clean up your articles, but it’s going to cost you; for such copy editing services an author must give up either all copyrights or pay many hundreds, if not thousands, of dollars for open access. One might think the generosity of Cureus’ follow-the-rules “self-serve” model for publishing, which on average requires about 30 minutes of extra work, would be appropriately valued by all authors. Sadly this is not always the case.

Therefore, Cureus has hereby implemented a new policy of banning (“blocking”) an article from any chance of being published, regardless of the quality of science, if an author fails (for whatever reason) to twice submit a manuscript that complies with our clearly published author guidelines. Moreover any author that submits two unique articles that fall short in this way will be forever banished from Cureus. There are plenty of other journals that are happy to copyedit sloppy work. Authors who produce such manuscripts are invited to send their articles to those publishers.

In Cureus’ community of trust and mutual respect, there is no room for users who fail to follow the rules. But if you are among the vast majority of users who comply with our guidelines, I promise that you will be amply rewarded for your consideration and cooperation. Thank you.

Cureus has been accepted for indexing in PubMed Central!

We are pleased to announce that Cureus has been accepted for indexing in PubMed Central® (PMC) (with citations added to PubMed). PMC is a free full-text archive of biomedical and life science journal literature operated by the U.S. National Institutes of Health (NIH).

We know how much our authors value PMC indexing. Knowing your published article has been indexed should result in relief and validation that your work will be available for the medical community to discover, read, discuss and cite.

Cureus indexed in PMC

Since the start of the year, we’ve published nearly 50 peer-reviewed articles documenting clinical experience and medical research from around the world. All of these articles can now be found in PMC (and their citations in PubMed), and we’re looking forward to the continued expansion of the Cureus library of peer-reviewed literature. Going forward, all articles published in Cureus will be indexed in PMC within one month of publication.

Thank you for your continued support of Cureus. This is a big step for our journal and we’re looking forward to more articles and more readers in the coming months. Please contact us at info@cureus.com with any questions.

Note (8/19/15): We’ve recently discovered that although articles published within Cureus are cited within PubMed, this does not constitute formal indexing. This blog post has been updated to reflect this distinction.