How might AI-assisted peer review tools reshape scientific publishing, and what safeguards should be implemented to maintain review quality while addressing the increasing volume of submissions?

The peer-reviewing process is being faced with ever-growing challenges as submission rates are still rising exponentially in all areas. Journal editors are faced with the challenge of getting capable reviewers who are ready to volunteer their time, with a result being delays in reviewing, reviewer exhaustion, and worse, compromised quality. Meanwhile, AI technologies are accelerating their ability to review scientific manuscripts for methodology, statistical fitting, plagiarism screenings, and even conceptual novelty detection.
1
Charles
At present all serious publishers (Wiley, Elsevier, Springer, MDPI, etc.) do not allow to submit the manuscript to be reviewed to AI. AI may produce very good answers mixed with shocking (badly wrong) ones. The latter can be filtered out by good reviewers, but dubious statements are impossible to detect. Language corrections are OK, but the different AI platforms provide different, partly contradictory, responses; hence, they are not scientific. The usage of AI would be helpful to detect plagiarism, but the success rate is not sufficient AND the newest AI versions can provide papers in your own styles, i.e. the development of AI will limit the detection of plagiarism.
I would not encourage giving money for a review. However, NO scientist should be allowed to publish a paper without making several COMPETENT reviews (say three/ submissions). As editor, I experienced that big scientists are rarely available for a review, if at all; some penalty should be included in the scientometric evaluation for that.
0
Jeff Erlich
Authors are already using AI to improve their papers (which is a good thing!). So, just like other fields, we should use AI to improve science, science communication and the process of going from submission to publication.

As AI gets better, I think journals should "pre-review" papers with AI to assist reviewers. The pre-review can:
  1. Summarise / list recent related work to evaluate novelty and impact of findings
  2. Point out potential statistical anomalies
  3. Create a table linking the main claims of the paper with figures/sections of the results and section of the methods.

These pre-reviews can also be part of the editorial assessment. I think with proper prompt engineering and fine-tuning, there is no good reason not to facilitate peer-review with AI. 

Of course, these need to be "air-gapped" or otherwise secured so submitted papers do not become part of training data. 

I find that peer review is a highly stochastic process. Sometimes it works well, sometimes not. Public post-publication review (i.e. like pubpeer) is the future. 
0
Agerie mengistie
very good

Post an Answer

Sign In to Answer