Toward Trustworthy AI in Higher Education: An Empirical Evaluation of GenAI in Management Education

Abstract

Recent advances in generative artificial intelligence
(GenAI) have significantly expanded the role of large
language models (LLMs) in higher education, offering new
opportunities for automated learning support and knowledge
dissemination. However, their accuracy in
discipline-specific fields such as management remains
underexplored. Existing studies often emphasize overall
accuracy while neglecting variations across question types,
such as definitional, case analysis, and problem-solving
tasks. To address this gap, we adopt a mixed evaluation
strategy that combines objective metrics (accuracy and
semantic similarity scores) with expert-based assessments
of correctness, logical coherence, and disciplinary
relevance. Our findings show that GPT-4 achieves the
highest overall accuracy and consistency, particularly
excelling in case-based reasoning. LLaMA performs
competitively on definitional and factual questions but
struggles with complex, multi-step reasoning, while
DeepSeek demonstrates efficiency and contextual relevance,
though with lower accuracy in advanced analytical
scenarios. These results highlight both the potential and
limitations of current LLMs in management education and
provide actionable insights for educators and curriculum
designers on integrating GenAI tools into higher education.