AI’s role in rampant scientific fraud undermines nutrition science

Artificial intelligence is making it easier to churn out fraudulent scientific research, some observers warn. It’s unclear whether nutrition science has been deeply compromised yet, but the threat is growing. 

The attention-grabbing headline in this area came in late December, when the journal Nature reported that a record-setting 10,000 papers were retracted in 2023. 

Hindawi was biggest offender 

The biggest chunk of those came from the publisher Hindawi, which publishes more than 250 open access journals, two of which are in the nutrition space, including Evidence-Based Complementary and Alternative Medicine as well as Oxidative Medicine and Cellular Longevity.  

The scientific publisher Wiley bought Hindawi in 2021, and when the issue of fraudulent papers came to light, Wiley formalized a process that saw about 8,000 papers retracted across all its journals in 2023. According to the site Retraction Watch, the debacle will cost the publisher $35 million to $40 million in lost revenue. Wiley reportedly will stop using the Hindawi name sometime this year. 

Sabine Hossenfelder, Ph.D., a theoretical physicist and YouTube science popularizer, recently said the papers were retracted “not because they contained honest mistakes but fabricated crap. Sham data, AI-generated text, repurposed figures and images.” 

Hossenfelder cited special issues as a source of particular concern. The rationale behind the publication of such issues has been to gather research on topics for which there might not be a dedicated journal and they therefore at least in theory served a legitimate purpose. 

However, Hossenfelder said such issues have become notorious because they are often farmed out to independent editors and the peer-review process can be circumvented. 

For example, Hindawi in 2023 published a special edition titled Pharmacological Potential of Natural Products Against Cardiovascular Diseases. The lead editor and one of the associate editors was based in Pakistan, with other associate editors based in Saudi Arabia and Romania. 

A full 50 papers from this special edition have been retracted. Many of the retracted papers show the same date for receipt and acceptance of the paper. 

In a statement, Wiley said these papers were retracted because its investigation “has uncovered evidence of systematic manipulation of the publication and peer-review process. We cannot, therefore, vouch for the reliability or integrity of this article. 

“Wiley and Hindawi regret that the usual quality checks did not identify these issues before publication and have since put additional measures in place to safeguard research integrity,” the statement added. 

Questions raised about Hindawi years ago 

Years before the Wiley acquisition, Hindawi had been identified as a “predatory publisher” by Jeffrey Beall, once an academic librarian and associate professor at the University of Colorado Denver. Beall developed a metric to determine which of the rapidly proliferating journals were most prone to publish shoddy research.  

From 2012 to 2017, Beall, who’s now retired, maintained “Beall’s List” that once included hundreds of journals. In an article published in 2017 in the journal Biochemia Medica, Beall described why he had to shut down the effort. He said the university administration had bowed to pressure from the publishers whose journals were on the list. The open access idea had become a huge cash cow, Beall said, one that the publishers were willing to defend vigorously. 

“The scholarly publishing industry is also responsible for its own decline . . . The industry has consistently failed to regulate itself. It allowed the predatory journals to appear, multiply, and prosper, and it looked the other way,” Beall wrote. 

Reviewers: The threat is real 

Natural Products Insider contacted two experts who have done peer review of nutritional science papers. Both said that fraudulent and/or shoddy research has always been an issue, but the vast proliferation of journals and the potential of AI to rapidly churn out valid-looking papers is a cause for concern. 

Ralf Jäger, Ph.D., head of the scientific consulting firm Increnovo, acts as associate editor for two scientific journals, Probiotics and Antimicrobial Proteins and the Journal of the International Society of Sports Nutrition (ISSN). He also frequently acts as a peer reviewer of papers that appear in other journals. 

He said he hasn’t come across a deliberately fraudulent paper yet, but he’s aware the threat is growing. Yet he said there’s cause for optimism, too. 

“With the advances in AI and the emergence of paper mills, there is an increasing concern about fabricated articles. On the flip side, AI will also help us to identify fabricated articles, too,” he said. 

Susan Kleiner, Ph.D., a nutrition researcher, consulting nutritionist and member of ISSN, said there is more variance in the quality of submitted manuscripts now than what she has seen in the past. 

“As a peer reviewer in our industry, I regularly see poorly designed and written manuscripts. It’s not always obvious if this is intentional or just poor guidance and review by professors/supervisors, especially from authors with English as a second language,” she said. 

“I am undoubtedly spending more time in recent years reviewing more basic problems with methodology and study design, statistics and analytics, discussion and conclusions, and even grammar, than ever before. I am definitely refusing to publish more manuscripts than I did historically,” Kleiner added. 

Reproducibility key to validity 

Jäger said a key point to remember is that even the most exciting result is open to debate until it has been replicated. A well-designed study worthy of publication should function as a blueprint of sorts for other scientists who can verify the results. 

“Scientific publications are intended to teach a person skilled in the art how to do a certain experiment, allowing other scientists to repeat and confirm, or dispute findings. In nutritional science, we have this very important safety valve. Scientific papers contain experiments, and those experiments can be repeated by other scientists to confirm or dispute findings,” he said. 

AI can be a legitimate tool 

Kleiner said using AI to help write the body of the paper is undoubtedly taking place. She said that’s not an issue so long as the researchers aren’t also relying on AI to determine the conclusions. And in any case, conclusions that step beyond what the data says is hardly a new concept. 

“Long before AI, it was not uncommon to see manuscripts where the data did not support the conclusions. The conclusions were written like marketing language that could be lifted out of the published study and put on advertising and marketing materials,” she said. 

“All of this is to say that there is a lot more pressure on reviewers to spend a serious amount of time in their roles, with absolute expertise in the area of research in the manuscript. It is harder to get reviewers today, and this is one reason why that may be,” Kleiner added.