America
As debate rages on the potential and risks of artificial intelligence (AI) tools such as ChatGPT to higher education, raters also asked what role AI and machine learning could play in that field. increase.
At a virtual symposium hosted by the Center for Research and Evaluation at the University of Mississippi, USA, on March 24, independent evaluation consultant Silva Ferretti described ChatGPT as a consummate bureaucrat.
Instead of worrying about AI replacing humans in the field of evaluation, she argued, we should be concerned about the extent to which humans are beginning to think like machines.
The symposium titled “Are we at a crossroads?” also explored the impact and opportunities of AI in assessment. It was hosted by Dr. Sarah Mason of the University of Mississippi and Dr. Bianca Montross Moorhead of the University of Connecticut. A new direction for evaluationa publication of the American Rating Society.
They said sectors around the world are grappling with the question of whether ChatGPT will mark a fork in the road when it comes to powerful new generative AI. “Generative AI differs from previous AI models in that it can create entirely new content, so this potential fork emerges.”
ChatGPT multiple roles
Speaker Silva Ferretti says using ChatGPT at work saves her a lot of time, not only gives her ideas, but even acts as a “great sparring partner for thinking.”
“You can take on the role of a feminist evaluator, a conventional evaluator, a technocratic expert, and many other different perspectives,” she says. “AI can offer a ‘by the books’ approach. [the human evaluator] Free time to move on and explore details, alternatives and possibilities. “
Ferretti explained how he uses ChatGPT for tasks such as creating concept notes, surveys, log frames, and metrics.
“For me, using ChatGPT really pushes me to another level. Why should people pay me when I’m trying to make something that machines can do?” Next year, ChatGPT may have better results.”
Another speaker, Dr. Tarek Azam, director of the Center for Evaluation and Evaluation at the University of California, Santa Barbara, argued that AI still has a long way to go before it becomes a real threat to human evaluators.
“We have been trained by many experiences over the years to be able to think about our results, our results, and make valid arguments to support the conclusions we come up with.” He said, “The main reason is all that is necessary to draw valid arguments and valid conclusions.”
But like Ferretti, Adzam sees room for AI in research evaluation.
“So, instead of having AI and/or humans, how do we actually leverage AI for both training and understanding?”
He gives one example put forward by the National Academy of Sciences to consider an automatic scoring scenario in which an AI engine takes the place of one of two human graders. “This is something that can be a big advantage from a time and cost standpoint,” he said.
Risks of ChatGPT
Ferretti said using ChatGPT comes with risks, one of which is that it tends to be hallucinogenic and simply hoax. This issue was repeated by several speakers throughout the event.
Dr. Izzy Thornton of the Center for Research Evaluation at the University of Mississippi pointed out the problem of algorithmic discrimination. This actually boils down to machine learning models being just as good as the data they were trained on.
As an example of how far models can go, Thornton talked about a machine learning model trained to distinguish images of malignant skin tumors from benign skin tumors. The model approached human accuracy in discriminating one type of tumor from another.
But when the researchers took the model apart to see how it worked, they found that the main factor they used to determine the malignancy of an image was whether it had a ruler. got it.
“The problem here is,” she explained. Most malignant tumors were photographed with a ruler next to them for scale, but most benign tumors were not. “
There is also the more sinister way that the training of these models reflects the biases and prejudices of the people who develop and train them. Here, Thornton pointed to a well-known example of self-driving cars failing to detect dark-skinned people more than light-skinned people.
She also spoke about concerns about the validity and reliability of AI models.
“Machine learning models are trained to spit out words that sound right in response to other things they know to be true. We can’t be sure that the results are actually valid because we can’t get them.”
Ferretti, too, has experienced this in his work with ChatGPT, noting that every point needs to be fact-checked, and reliance on its own accuracy can lead to false reliance on humans as well. Yes, and I actually do.
Issues of fairness in the use of AI
Eileen Reed, Ph.D., of the Department of Education and Research Methodology at the University of North Carolina at Greensboro, says that with every technological advance comes concern.
There is a growing body of evidence showing how algorithms and AI can contribute to new modes of racial profiling and gender agnosticism, favoring the privileged classes of society while harming the poor and perpetuating injustice. I’m here.
“Knowing this, to what extent can a computer make ethical decisions? And to what extent should we trust it?”
But what all the speakers agreed on is that AI is not going anywhere. The options are to actively engage in the implementation of AI in the field of assessment, or to introduce AI into this field in a free and uncontrolled manner with unintended and potentially harmful consequences.
Reid described this as an opportunity for evaluators. Internet is unregulated. This is a public policy issue that will ultimately come under scrutiny. So, as a field, you have to create and create conversations to anticipate these intended and unintended effects, and really put yourself at the table while things develop.
