A new report says there is “widespread skepticism” among academics and experts about the use of AI to assess the quality of research.
The report, led by the University of Bristol and funded by Research England, shows that some universities are already using generative AI for this purpose, but that “national oversight and governance” is needed.
The UK system for assessing the quality of research in higher education institutions is known as the Research Excellence Framework (REF). The outcome will influence how around £2 billion of public funding a year is allocated to university research.
The last REF was held in 2021 and changes to guidance for the next REF2029 are expected to be announced this month. The total cost of REF2021 is estimated to be around £471m, averaging £3m per participating higher education institution, with REF2029 finances expected to be even higher.
Richard Watermyer, professor of higher education at the University of Bristol and lead author of the report, said: “GenAI has the potential to revolutionize national research assessment, contributing to the creation of a more efficient and level playing field. “While there has been vocal opposition to its inclusion in the REF, our report makes clear that GenAI tools are now widely used, albeit quietly, and that there is high expectation for their use by REF panelists.”
“This report is timely given the immense financial pressures facing the sector. It is widely accepted that the REF’s regulatory burden is high and will likely only increase. While nAI has the potential to alleviate some of this problem, it does not provide a complete solution and may also create new bureaucratic challenges of its own, including establishing new requirements and protocols for proper use.
This report looked at the use of GenAI in 16 higher education institutions, including Russell Group universities and recently established universities across the UK. The findings showed evidence of widespread adoption of GenAI to prepare REF submissions in some form. However, their scope and usage varied widely. The survey also included a survey of approximately 400 academics and professional services staff, asking them how they felt about GenAI tools being used for various aspects of REF2029.
All aspects are shown to be strongly opposed by the majority of academics and professional services staff, with levels of strong opposition to different parts of the REF process varying between 54% and 75% of respondents. Almost a quarter (23%) of respondents were most supportive of GenAI tools being introduced to assist universities in developing impactful case studies.
The report makes a number of recommendations, including that all universities should establish and publish policies regarding the use of GenAI for research purposes, including REF. Relevant staff must be fully trained in the responsible and effective use of AI tools. and appropriate security and risk management measures must be implemented.
It also calls for strong national oversight, including sector-wide guidance on the use of REF29 and a comprehensive REF AI governance framework. To achieve equitable access to technology across all higher education institutions, it is recommended that a shared high-quality AI platform for REF be developed and made accessible to all institutions.
Get all the latest industry news and analysis – subscribe to Research Information Newsline!
