⇧ [VIDÉO] You might also like this partner content
The most advanced language models to date, such as OpenAI’s GPT-4, are becoming so good that it is now difficult to distinguish them in writing. GPT-4 notably obtained a score of 54% (success) during a recent test where it had to pass itself off as a human during a five-minute conversation (the Turing test). However, the rapid evolution of these systems seems to be becoming very problematic for the educational field. This is what has just been proven by an experiment carried out by researchers at the University of Reading aimed at secretly infiltrating university exams to have them administered by an AI.
During the COVID-19 pandemic, many universities around the world have switched from on-site supervised exams to unsupervised at-home exam sessions. Today, even though there is no longer a lockdown, this practice continues, and this is what worries teachers and researchers.
Indeed, with AI tools becoming more and more powerful and accessible, students can easily cheat by submitting work generated entirely or largely by them. Especially since the situation has become increasingly complicated with the decline in reliability of AI-based text detection tools, which have become ineffective most of the time.
A real-life test
To measure the extent of the problem for the education sector, particularly within universities and colleges, researchers carried out an “infiltration test”. With his colleagues, Peter Scarfe, professor and researcher at the University of Reading, played with the establishment’s exam markers to see if they would be able to distinguish answers generated by an AI from those written by real students. To do this, they took part in a psychology exam at their establishment and provided answers entirely produced by GPT-4 (OpenAI’s most powerful language model) under 33 false student names.
« Many institutions have moved away from traditional exams to make assessment more inclusive. Our research shows that understanding how AI will affect the integrity of educational assessments is of international importance “, Scarfe said. This blind study, aimed at challenging human educators to detect AI-generated content, is the largest of its kind conducted to date.
Clear results
The results of the study, published in the journal PLOS
ONE on June 26, are final. 94% of responses that were generated by Chat GPT, submitted for several undergraduate psychology modules, were not unmasked (as coming from an AI). Even more surprising, in 83.4% of cases, AI submissions received higher grades than (randomly selected) students. However, Scarfe and his team couldn’t rule out the fact that these students too might have used AI to answer the questions.
See as well
These results thus highlight two major problems. On the one hand, there is the high probability that students will use AI to cheat without it being detected. On the other hand, they may get better grades than others who would not have cheated. “ The publication of this real-world quality assurance test clearly demonstrates that freely and openly available generative AI tools make it easy for students to cheat on take-home exams to get better grades, while the cheating is undetectable. ” said Karen Yeung, a professor at the University of Birmingham who specializes in law, ethics and computer science.
To effectively address these issues, Scarfe suggests simply returning to in-person supervised exams. Professor Elizabeth McCrum, vice-chancellor for education and student experience at the University of Reading, disagrees. She said: It’s clear that AI will have a transformative effect in many aspects of our lives, including how we teach students and assess their learning. However, solutions include moving away from outdated ideas around assessment and towards ideas more relevant to the skills students will need in the workplace, including using AI “, concludes McCrum.
