University uses software to detect AI at student work

Several universities have used software to detect the use of artificial intelligence in student work.

Massey University recently made a decision, and Auckland and the University of Victoria also did not use it.

In Massey's case, after a massive technical failure last year, he followed the decision to stop using automated systems to monitor fraud in online exams.

RNZ detection was unreliable, allowing students to use AI responsibly for much of their work anyway.

One of the presidents of Dr. Angela Feekelly, Massey's Higher Education Union division, said RNZ scholars have not consistently used AI detection.

Some people used the results as guidelines, but others would accuse students of being cheated if they suggest that their work consists of a certain percentage of AI-generated content.

“There have been a lot of research that essentially announces that AI detection doesn't work too well. There are so many tools that students can use to see if their work is detected by AI, and they can fool it anyway,” she said.

“There's been almost a decision to turn it off because it's not effective.”

Dr. Feekery said there are other ways to find AI use, such as checking the history of the document's version to see if it was created in two minutes rather than a few days, or simply using professional judgment.

“I've been teaching for 25 years. I've marked writing for students over the years. I know what it looks like. It's not what they're submitting now. In many cases, there's a problem when there's a student who can write better than the first year.”

Feekery said scholars are still trying to figure out the best ways to assess students in the AI ​​era.

“We don't have a solution yet, but there are certainly many conversations and students are at the centre of those conversations. We can keep in mind that our student experience is at the centre of the conversations we have about this,” she said.

Auckland University graduate teaching assistant Java Grant was holding a meeting on AI for the Higher Education Union next month.

He said Massey's decision made sense from a technical standpoint.

“It's really hard to distinguish between what could be generated by AI tools, unless there's a narrative indication like, 'I'm a big linguistic model so I can't answer this,'” he said.

He said many scholars have chosen to use forms of assessment that do not allow AI to be used.

“There's so much sensitivity to mistakenly accusing students of using tools, and it's the best solution I've found at the course level right now, and I personally went straight to paper tests to see how instructors and tutors work together to make sure students are learning content, but gal-stretched workloads.”

Dr. Ulrich Spidell, a senior lecturer in computer science at Auckland University, said relying on student integrity for remote assessments is open to abuse.

“Absolutely there's absolutely nothing to stop a second device floating or having a friend or helper. In such an exam, 30-60% of the class will appear to be useful for illegal help, depending on the class and background and demographics of the class,” he said.

Dr. Speidel said the figures were based on his experience and research.

However, he said automatic monitoring of digital exams could be hacked as they appear to be able to secure on-campus digital exams.

Speidel said Auckland discussed the use of automated AI detection several years ago and decided it was worthless because he couldn't clearly prove whether students used AI for their work.

Massey University said online ratings, such as online essay submissions and quizzes, have not been activated.

“These are part of a broader assessment approach that ensures that student work is properly validated at a critical point in the study,” he said.

“The impact of Generated Artificial Intelligence (GENAI) means that all universities are reviewing their approach to assessment. Like many other universities, Massey no longer uses AI detection because serious concerns have been raised about the reliability of the approach.

“As opposed to using unreliable detection tools, universities prioritize precautions such as ensuring assessments of unauthorized ratings. As part of this process, Massey is currently in the process of consultation to determine future approaches to exam delivery.”

Massey said students were allowed to use AI in all assessments, except for those that could be secured in a way that prevents them, rather than detecting AI usage.

These include laboratory and studio-based activities, oral assessments, and exams.

“Being away from detection doesn’t mean we simply delegate thought, reasoning, and strict academic practice to AI, but rather we show that we are aware that our environment is changing and we must adapt accordingly.

“We are working to develop AI literacy across the university, so we will effectively support students in using AI as part of our academic toolkit and engage with it in a way that maintains ethical, learning-centric, and academic integrity.”

Approach to AI

Eight universities have online exam security and how AI detection is detected in student work:

Auckland

• Remote exams are encouraged online.
• We do not recommend AI detection tools.

Automatic

• Remote online exams will not be performed.
• It is unclear whether students are using AI detection software for their work.

Waikato

• We will conduct some exams online and some remotely.
• Use the AI ​​writing detection tool.

Massey

• Provides evaluation and testing of remote online open books without automatic monitoring.
• We do not use the software to check the use of AI in students' work.

Canterbury

• Use monitoring tools for online assessments.

Lincoln

• Monitor remote online exams using video conferencing technology.
• Use the software to check the use of AI at student work.

Victoria

• We rarely use digital exams and don't use online suggestions.
• It does not use AI detection.

Otago

• There are very few digital exams.
• Although it uses plagiarism detection software, RNZ understands that some scholars do not use AI detection capabilities.

Sign up for NgāPitopito Kōrero, A daily newsletter curated by our editors will be delivered directly to your inbox every week.