AI discovers newness: Jülich team wins global competition

June 8, 2026

A research team from Forschungszentrum Jülich has won the international “Metascience Novelty Indicators Challenge”. Scientists at Jülich Systems Analysis have developed a method that allows artificial intelligence to assess the novelty of scientific publications, that is, the extent to which research advances scientific knowledge. Due to this success, the team was awarded a prize of £300,000 to further develop the method.

The challenge was organized by the UK Metascience Unit (UKRI) in collaboration with international partners. The aim was to develop a scalable method for assessing the novelty of research papers at the time of publication.

For this purpose, the organizers provided a dataset of 100,000 recent scientific publications. Experts in their fields independently assessed each novelty. The task for the participating teams was to predict expert judgments as accurately as possible without knowing the ratings.

Urich’s approach achieved the best results across all evaluation criteria.

“Until now, human experts have been limited to the ability to assess what is truly novel and valuable in scientific papers,” says Dr. Ing. Jann Michael Weinand, Head of Integrated Scenarios, Institute for Climate and Energy Systems – Jülich System Analysis (ICE-2). “Our approach shows that modern AI systems can support this task with remarkable reliability.”

AI analyzes content, not citation counts

Unlike many established research metrics, the Jülich system does not assess how often a paper is later cited. “Metadata is not sufficient to assess novelty at the time of publication. Therefore, our system looks at the content of the research and relates it to the state of knowledge at the time of publication,” says fellow ICE-2 project leader Jan Gepfert, who developed the approach with colleague Samuel Keeling.

To do this, the system first analyzes the study itself and the selected scientific papers it references. Based on this, AI reconstructs the state of knowledge at the time of publication, including known gaps in research. Next, we evaluate the contributions of new research. Will new methods be introduced? Will the result be surprising? Will it solve a previously unsolved problem? The system intentionally collects arguments both for and against the paper’s novelty and weighs them against each other.

Ultimately, the AI ​​assigns a novelty score between 0 and 100. It also provides an interval that indicates how confident you are in evaluating the model. Detailed written justification makes the evaluation transparent. “The biggest challenge was to define novelty in a meaningful way. For us, novelty means more than just difference; what’s important is the work’s contribution to scientific progress,” Keeling says.

Early visualization of important research

The number of scientific publications continues to grow rapidly. At the same time, more papers are being produced using AI tools. This makes it increasingly difficult for researchers, journals, and funding bodies to identify particularly relevant contributions at an early stage.

This is where novelty indicators could be useful in the future. Research that is particularly likely to generate new insights may be identified during the peer review and publication process, rather than its importance becoming apparent through citation metrics years later.

“We hope this will be particularly useful for research that is often overlooked by traditional metrics,” Keeling says. “Our goal is not to replace human judgment; rather, AI should help draw attention to potentially important research and support better-informed decision-making.”

Moreover, novelty indicators bring new possibilities to metascience, i.e., the scientific study of the research system itself.

Prize money allows further development

With the £300,000 prize, the team intends to further develop the existing prototype into a reliable scientific tool. Novelty indicators are intended to be transparent and resistant to manipulation, and must not exacerbate existing inequalities within the scientific system.

In the long term, researchers envision applications far beyond scientific publications, such as in the patent context and identifying new and promising research questions and hypotheses. “At the same time, this development raises fundamental questions: What role should AI play in scientific decision-making in the future? And how can scientific evaluation and progress be kept transparent and traceable?” Goepfert says.

The work of the Jülich researchers shows that AI is capable of far more than just analyzing data and summarizing text. Scientific research itself can now be evaluated, opening up new possibilities for tomorrow’s science.

About the Metascience Novelty Indicator Challenge

The Metascience Novelty Indicators Challenge was hosted by UK Research and Innovation’s (UKRI) UK Metascience Unit and Coefficient Giving. Partners in the competition are the Science Policy Research Unit (SPRU) at the University of Sussex, RAND Europe, a research and consultancy organization, and Challenge Works, a world leader in the design and delivery of challenge prizes and part of the research and innovation foundation Nesta. The £300,000 prize was provided by Coefficient Giving.