Reconfiguring Assessment in the Age of AI: Making Learning Visible through Meta-Task Awareness (MTA)

Abstract

The widespread use of large language models (LLMs) is
reshaping assessment practices, making it increasingly
difficult to determine what learners actually know and can
do, as learning processes become less visible in
AI-mediated work. Rather than treating AI use as a problem
of restriction or detection, this study explores how
assessment can be reconfigured to make learning visible
through learners’ interaction with AI. Focusing on a
postgraduate teacher education course, the study introduces
Meta-Task Awareness (MTA) as an analytic lens for examining
how learners regulate task goals, pedagogical
considerations, and epistemic responsibility in
LLM-supported lesson design. Using an exploratory
qualitative design with descriptive code-and-count
summaries, interaction trace data from 11 students were
analysed at the episode level using a theory-informed
coding framework, with over 90% agreement across coded
instances. Findings show that students’ engagement was
primarily oriented toward task clarification, pedagogical
reasoning, and evaluative judgment, with varying patterns
resulting in three distinct orientations toward AI-mediated
design. The study shows how assessment can be reconfigured
so that learning becomes more visible, more accountable,
and increasingly unavoidable in the age of AI.