Using AI in science can add to reproducibility woes, say boffins

Using AI in science promises to add to problems in reproducing important results, the UK's highly prestigious Royal Society has warned.

In its report "Science in the age of AI," the 350-year-old institution argues that introducing AI in scientific research created barriers to reproducibility - the idea that a particular result can be replicated by a different research team in another part of the world - through lack of documentation, limited access to essential computing infrastructures and resources, and a struggle to understand how AI tools reach their conclusions.

The tech industry has enthusiastically promoted the idea that AI can help in science. In December last year, researchers claimed to have made the world's first scientific discovery using large language models - a breakthrough suggesting LLMs like ChatGPT could push science forward faster than humans alone.

However, professor Alison Noble, chair of the Royal Society Science in the Age of AI Working Group, is worried the rapid uptake of AI in science has presented challenges related to its safe and rigorous use.

"A growing body of irreproducible studies are raising concerns regarding the robustness of AI-based discoveries," she declared.

AI limits the ability to reproduce results owing to the proprietary nature of many AI tools.

"Barriers such as insufficient documentation, limited access to essential infrastructures (eg code, data, and computing power) and a lack of understanding of how AI tools reach their conclusions (explainability) make it difficult for independent researchers to scrutinise, verify and replicate experiments," the paper explains.

The paper warns that reliance on AI in research could lead to "inflated expectations, exaggerated claims of accuracy, or research outputs based on spurious correlations."

"In the case of AI-based research, being able to reproduce a study not only involves replicating the method, but also being able to reproduce the code, data, and environmental conditions under which the experiment was conducted (eg computing, hardware, software)."

The Royal Society was founded in 1660 and scientists including Isaac Newton, chemist Humphry Davy, and Ernest Rutherford, discoverer of the atomic nucleus, are among its past presidents.

In its AI paper, the Society warns reproducibility failures not only risk the validity of the individual study, but could skew subsequent research.

The study, led by the Center for Statistics and Machine Learning at Princeton University, shows how "data leakage" in a single research project - a leading cause of errors in ML applications - may affect 294 papers across 17 scientific fields, including high-stakes fields like medicine.

Models developed in a commercial setting can add to the problem. "For instance, most leading LLMs are developed by large technology companies like Google, Microsoft, Meta, and OpenAI. These models are proprietary systems, and as such, reveal limited information about their model architecture, training data, and the decision-making processes that would enhance understanding."

To address these challenges, scientists should adopt open science principles - for example, the UNESCO Recommendation on Open Science. The study also suggests that grand challenges - such as the ML Reproducibility Challenge, which invites participants to reproduce papers published in 11 top ML conferences - could help.

In August last year, researchers warned poor data quality was also a problem in AI-based research, while difficulties reproducing results assisted by AI arose from the random or stochastic approach to training deep learning models.

The Stanford computer science team argued that standardized benchmarks and experimental design can alleviate such issues.

"Another direction towards improving reproducibility is through open source initiatives that release open models, datasets and education programmes," the Royal Society's research paper adds. ®

About Us
Website HardCracked provides softwares, patches, cracks and keygens. If you have software or keygens to share, feel free to submit it to us here. Also you may contact us if you have software that needs to be removed from our website. Thanks for use our service!
IT News
Jun 17
Nearly 20% of running SQL Servers have passed end of support

Exclusive That beige box running a server is easily forgotten ... until it goes wrong

Jun 17
Can platform-wide AI ever fit into enterprise security?

Opinion You know what they say about headlines that end in a question mark

Jun 17
Notorious cyber gang UNC3944 attacks vSphere and Azure to run VMs inside victims' infrastructure

Who needs ransomware when you can scare techies into coughing up their credentials?

Jun 17
China's Big Tech companies taught Asia to pay by scanning QR codes, but made a mess along the way

Feature A push for interoperability is accelerating, but maybe not fast enough to stop biometrics taking over

Jun 16
From RAGs to riches: A practical guide to making your local AI chatbot smarter

Hands on Nine out of 10 execs recommend adding Retrieval Augmented Generation to your daily regimen

Jun 15
European Commission may be about to put the squeeze on Apple for its App Store rules

iBiz potentially facing hefty penalties under the Digital Markets Act