IoT Implementation Learning Support System Integrating AR Multimodal Interaction and Generative AI

Abstract

In IoT implementation courses, students often face high
cognitive load due to switching attention between materials
and circuits. While Augmented Reality (AR) offers visual
guidance, handheld devices cause "hands-occupied" issues
and lack real-time semantic feedback. This study proposes
the design of an intelligent support system integrating AR
smart glasses with the Google Gemini Large Language Model.
The proposed system architecture overlays circuit
information onto physical components via AR glasses,
enabling a hands-free environment. By combining voice
recognition with Generative AI, it is designed to provide
real-time debugging assistance and theoretical scaffolding.
This multimodal interaction approach is expected to reduce
extraneous cognitive load and enhance problem-solving
efficiency in IoT learning contexts.