Exploring the Effect of Scaffolding Strategies in GenAI Chatbot on Student Engagement and Programming Skill Development

Abstract

Generative artificial intelligence (GenAI) technology has been integrated into various educational contexts since its introduction. However, research in this field often focuses more on GenAI’s impact on students’ learning outcomes rather than the instructional strategies employed to support learning. This study explores the effects of scaffolding strategies used by a GenAI educational chatbot on two aspects of learning: behavioral engagement with the chatbot and improvements in programming skills. 52 students interacted with different versions of a GenAI programming educational chatbot, varying in the extent of scaffolding provided. Their programming skills were assessed and compared before and after the intervention. A thematic analysis of the topics discussed in, and cognitive complexity of students’ questions was also conducted. The study found that strategies which focused on enhancing conceptual understanding, as well as those that guided reflective practices, effectively fostered engagement, critical thinking, and programming skill development. These results underscore the need to align GenAI tools’ functionality with students’ needs to support meaningful learning. This study offers insights into the design of GenAI educational tools, opening paths for future research on training GenAI models to implement teaching pedagogies effectively.