At the outbreak of COVID-19, the World Health Organization warned that we were in the midst of an “infodemic”.  They were especially concerned about the spread of misinformation undermining our attempts to successfully tackle the virus.  New research from Carnegie Mellon suggests we have experienced an infodemic of a different kind over the past year, with the flurry of research into the pandemic making it very hard for even the keenest observer to keep up.

What’s more, this rush to publish may also have resulted in missteps being made, with the authors arguing that attention needs to be given to ensure that the most relevant and reliable information is properly recognized.

The authors suggest that possible solutions might combine AI with human expertise to ensure that we can keep pace with the knowledge base as it grows.  For instance, AI might collect and summarize research, with humans then curating the findings.

“Given the ever-increasing research volume, it will be hard for humans alone to keep pace,” they explain.

Rushing things

The authors suggest that the unique circumstances posed by COVID-19 might be prompting scientists to rush things due to the intense desire from clinicians to offer the best care to their patients.  This has resulted in over 8,000 preprints of scientific papers related to the virus being published by mid-August.  Even more have been added on issues related to the virus.

This has coincided with an average time to conduct the peer review process and publish each article shrinking, with virology papers being turned around in just 60 days.  The World Health Organization undoubtedly focused on deliberately false information when warning of the “infodemic”, but with papers around topics such as hydroxychloroquine being rushed to publication and then withdrawn, the scientific community is not without blame.

“We’re going to have that same conversation with vaccines,” the authors caution. “We’re going to have a lot of debates.”

As with so much, the problems around scientific publishing are not new but have certainly been exacerbated by the pandemic.  The authors propose various approaches to improve matters, such as a rise in the publication of things that haven’t worked as well as those that have to help prevent the scientific community from repeating past mistakes.

They also believe that AI could be used to digest and consolidate research.  To date, attempts to use AI for such ends have floundered because of the figurative and ambiguous language used by researchers is not easy for AI systems to decipher.  As such, they argue that it might be necessary for researchers to write one paper for humans and one for machines.

While the paper is by no means an end to the debate, the authors nonetheless hope that it will provide a worthwhile addition to the literature at a time when the reliability of scientific publication is more important than ever.

“Putting such infrastructure in place will help society with the next strategic surprise or grand challenge, which is likely to be equally, if not more, knowledge intensive,” the authors conclude.