Automated Misconception Diagnosis & Pedagogical Remediation: Using Generative AI to Identify and Remedy Structural Bugs in Introductory Code

Author

Ms. Bhanupriya Sharma


Abstract

 Novice programmers in introductory computer science (CS1) courses frequently experience cognitive overload, helplessness, and attrition due to structural code bugs. Unlike superficial syntax errors caught by standard compilers, structural bugs stem from fundamental misalignments between a student's mental model and the runtime execution context. While recent advances in Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) demonstrate remarkable code generation capabilities, generic LLM feedback often defaults to providing direct, answer-giving code patches. This passive resolution deprives students of essential problem-solving opportunities and exacerbates misconception persistence. This paper introduces StructDiag-AI, a novel novel Socratic diagnostic framework that integrates Abstract Syntax Tree (AST) static analysis with constrained LLM prompt synthesis. By isolating AST structural divergences prior to prompt assembly, StructDiag-AI identifies non-viable algorithmic patterns (e.g., misnested loops, unreachable recursive branches, and state mutation outside iteration) and generates progressive, non-generative Socratic pedagogical hints. We present a multi-institutional empirical trial ($N = 482$ CS1 students) evaluating StructDiag-AI against traditional compiler errors and baseline unconstrained LLM feedback. Quantitative results demonstrate that StructDiag-AI increases novice bug resolution rates by 38.4% compared to standard compilers and improves long-term conceptual retention by 24.1% over unconstrained LLM feedback. Furthermore, qualitative analysis reveals a significant reduction in student frustration metrics and rapid convergence toward correct mental models.



Keywords

Providing Socratic, AST-guided structural feedback reduces novice resolution time by 42% while significantly preventing code-dependency traps common in unconstrained LLM assistance.



Full Text:

Download Paper PDF


References


  1. Becker, B., Denny, P., Finnie-Ansley, J., Luxton-Reilly, A., Prather, J., & Santos, E. A. (2023). Programming Is Hard - Or Is It? This AI Just Solved Your Whole Course. Proceedings of the 54th ACM Technical Symposium on Computer Science Education (SIGCSE 2023), 510–516.
  2. Denny, P., Luxton-Reilly, A., Leinonen, J., & Hellas, A. (2024). Prompt Problems: A New Type of Programming Exercise for the Generative AI Era. ACM Transactions on Computing Education (TOCE), 24(1), 1–28.
  3. Kazerouni, A., Shaffer, C. A., Clifford, C., & Tilevich, E. (2019). Quantifying the Impact of Devised Scaffolding on Novice Debugging Performance. IEEE Transactions on Education, 62(3), 194–202.
  4. Prather, J., Reeves, B. N., MacNeil, S., & Leinonen, J. (2023). What Do Students Ask AI? Analyzing Student Prompts in LLM-Assisted CS1. Proceedings of the 2023 ACM Conference on International Computing Education Research (ICER 2023), 142–155.
  5. Qian, Y., & Lehman, J. (2017). Students' Misconceptions and Other Difficulties in Introductory Programming: A Literature Review. ACM Transactions on Computing Education (TOCE), 18(1), 1–24.
  6. Robins, A., Rountree, J., & Rountree, N. (2003). Learning and Teaching Programming: A Review and Discussion. Computer Science Education, 13(2), 137–172.
  7. Smith, J., & Wang, L. (2025). AST-Driven Pedagogical Feedback Engines in Automated CS Assessment. Journal of Educational Data Mining, 17(2), 88–112.

Share your valuable work from Social Media Buttons