AI-detection software isn’t the solution to classroom cheating — assessment has to shift

This article was originally published on The Conversation, an independent and nonprofit source of news, analysis and commentary from academic experts. Disclosure information is available on the original site.

Author: Michael Holden, Assistant Professor, Faculty of Education, University of Winnipeg

Two years since the release of ChatGPT, teachers and institutions are still struggling with assessment in the age of artificial intelligence (AI).

Some have banned AI tools outright. Others have turned to AI tools only to abandon them months later or have called for teachers to embrace AI to transform assessment.

The result is a hodgepodge of responses, leaving many kindergarten to Grade 12 and post-secondary teachers to make decisions about AI use that may not be aligned with the teacher next door, institutional policies, or current research on what AI can and cannot do.

One response has been to use AI detection software, which rely on algorithms to try to identify how a specific text was generated.

AI detection tools are better than humans at spotting AI-generated work. But they’re a sufficiently imperfect solution, and they do nothing to address the core validity problem of designing assessments where we can be confident in what students know and can do.

Teachers using AI detectors

A recent American survey, based on nationally representative surveys of K-12 public school teachers published by the Center for Democracy and Technology, reported that 68 per cent of teachers use AI detectors.

This practice has also founds its way into some Canadian K-12 schools and universities.

AI detectors vary in their methods. Two common approaches are to check for qualities described as “burstiness,” referring to alternating and short and long sentences (the way humans tend to write) and complexity (or “perplexity”). If an assignment does not have the typical markers of human-generated text, the software may flag it as AI-generated, prompting the teacher to begin an investigation for academic misconduct.

To its credit, AI detection software is more reliable than human detection. Repeated studies across contexts show humans — including teachers and other experts — are incapable of reliably distinguishing AI-generated text, despite teachers’ confidence that they can spot a fake.

.

.

.

Need to change assessment

The solution to taller cheating ladders is not taller walls. The solution is to change how we are assessing — something classroom assessment researchers have been advocating for long before the onset of AI.

Just as we don’t spend thousands of dollars on “did-their-sister-write-this” detectors, schools should not rest easy simply because AI detection companies have a product to sell. If educators want to make valid inferences about what students know and can do, assessment practices are needed that emphasize ongoing formative assessment (like drafts, works-in-progress and repeated observations of student learning).

These need to be rooted in authentic contexts relevant to students’ lives and their learning that centre comprehensive academic integrity as a shared responsibility of students, teachers and system leaders — not just a mantra of “don’t cheat and if we catch you we will punish you.”

Let’s spend less on flawed detection tools and more on supporting teachers to develop their assessment capacity across the board.

Read entire article here.