AI-Based Language Models (LLMs) in Assessing ESL Learners’ Speaking Skills: Opportunities and Limitations

Author

Leyla Hajiyeva


Abstract

 Speaking is widely regarded as the most difficult skill to teach, practise, and assess in English as a Second Language (ESL) classrooms, largely because valid assessment of oral proficiency has traditionally depended on scarce resources: trained human raters, controlled testing conditions, and considerable time. The recent emergence of large language models (LLMs) capable of real-time speech-to-text conversion, conversational interaction, and rubric-referenced feedback generation has renewed interest in automating parts of this process. This article critically reviews the opportunities and limitations of LLM-based tools for assessing and supporting the development of ESL learners’ speaking skills. It traces the evolution from earlier automatic speech-scoring systems grounded in automatic speech recognition (ASR) and statistical feature extraction to the current generation of generative, instruction-following language models capable of producing holistic and analytic feedback in natural language. Five categories of opportunity are identified — scalable formative feedback, on-demand conversational practice, rubric-referenced scoring support, fine-grained pronunciation and fluency diagnostics, and reduced speaking anxiety through low-stakes practice — alongside five categories of limitation: construct underrepresentation, accent- and dialect-related bias, hallucination and inconsistent reliability, washback effects and learner overreliance, and unresolved data-privacy and ethical concerns. Building on this synthesis, the article proposes a five-criterion framework — validity, reliability, fairness, transparency, and pedagogical alignment — intended to guide the responsible integration of LLM-based tools into ESL speaking instruction and assessment, and discusses implications for English-language teaching contexts such as Azerbaijan, where access to trained oral examiners is often constrained. The article concludes that LLMs are best positioned as a complement to, rather than a replacement for, human assessment of spoken English, and it outlines a research agenda for validating LLM-based speaking assessment against established testing standards.



Keywords

Large language models, ESL speaking assessment, Automated speech scoring, Artificial intelligence in language education, Formative feedback, Oral proficiency, Second language testing



Full Text:

Download Paper PDF


References


  • Horwitz, E. K., Horwitz, M. B., & Cope, J. (1986). Foreign language classroom anxiety. The Modern Language Journal, 70(2), 125–132.
  • Luoma, S. (2004). Assessing speaking. Cambridge University Press.
  • Zechner, K., Higgins, D., Xi, X., & Williamson, D. M. (2009). Automatic scoring of non-native spontaneous speech in tests of spoken English. Speech Communication, 51(10), 883–895.
  • Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274.
  • Hajiyeva, L. (2025). Advantages and disadvantages of AI in modern ESL classes. The Scientific Heritage, (166), 62–64. https://doi.org/10.5281/zenodo.16763524
  • Hajiyeva, L. (2024). Analysing of the commonly used English idioms and their role in learning English. Qədim Diyar, 25, 25.
  • Seyidzadeh, L. E. (2019). Semantics of some slangs used in everyday speech in English with cognition.
  • Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge University Press.
  • Isaacs, T., & Thomson, R. I. (2013). Rater experience, rating scale length, and judgments of L2 pronunciation: Revisiting research conventions. Language Assessment Quarterly, 10(2), 135–159.
  • Van Moere, A., & Downey, R. (2016). Technology and artificial intelligence in language assessment. In D. Tsagari & J. Banerjee (Eds.), Handbook of second language assessment(pp. 341–358). De Gruyter Mouton.
  • Litman, D. (2016). Natural language processing for enhancing teaching and learning. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1), 4170–4176.
  • Godwin-Jones, R. (2022). Partnering with AI: Intelligent writing assistance and instructed language learning. Language Learning & Technology, 26(2), 5–24.
  • Warschauer, M., Tseng, W., Yim, S., Webster, T., Jacob, P., Du, Q., & Tate, T. (2023). The affordances and contradictions of AI-generated text for second language writing. Journal of Second Language Writing, 62, 101071.
  • Long, M. H. (1996). The role of the linguistic environment in second language acquisition. In W. C. Ritchie & T. K. Bhatia (Eds.), Handbook of second language acquisition(pp. 413–468). Academic Press.
  • Bibauw, S., François, T., & Desmet, P. (2019). Discussing with a computer to practice a foreign language: Research synthesis and conceptual framework of dialogue-based CALL. Computer Assisted Language Learning, 32(8), 827–877.
  • Bearman, M., & Luckin, R. (2020). Preparing university assessment for a world with AI: Tasks for human intelligence. In M. Bearman, P. Dawson, R. Ajjawi, J. Tai, & D. Boud (Eds.), Re-imagining university assessment in a digital world(pp. 49–63). Springer.
  • Chapelle, C. A., & Voss, E. (2016). 20 years of technology and language assessment in Language Learning & Technology. Language Learning & Technology, 20(2), 116–128.
  • Hajiyeva, L. E. (2026). Pseudonym as an integral element in the works of Azerbaijani, English and American writers and poets. Baltic Journal of Legal and Social Sciences, (2), 479–485. https://doi.org/10.30525/2592-8813-2026-2-57
  • Imamguluyev, R. (2020, July). Application of fuzzy logic model for correct lighting in computer aided interior design areas. In International Conference on Intelligent and Fuzzy Systems(pp. 1644–1651). Cham: Springer International Publishing.
  • Abdullayev, T., Imamguluyev, R., & Umarova, N. (2021, August). Application of fuzzy logic model for optimal solution of light reflection value in lighting calculations. In International Conference on Theory and Application of Soft Computing, Computing with Words and Perceptions(pp. 384–391). Cham: Springer International Publishing.
  • Imamguluyev, R., Mikayilova, R., & Salahli, V. (2022). Application of a fuzzy logic model for optimal assessment of the maintenance factor affecting lighting in interior design. In Mobile Computing and Sustainable Informatics: Proceedings of ICMCSI 2022(pp. 463–474). Singapore: Springer Nature Singapore.
  • Imamguluyev, R., Elvin Balakishiyev, & Nihad Agakishiev. (2020). IMPROVING THE MECHANISM OF USING THE PRICE FACTOR IN THE EFFECTIVE REGULATION OF AGRICULTURAL PRODUCTION ON THE BASIS OF FUZZY LOGIC .International Journal of Innovative Technologies in Economy, 5(32). https://doi.org/10.31435/rsglobal_ijite/30122020/7325.
  • Imamguluyev, R., Sadiqova, L., Aliyeva, A., & Mammadhasanov, E. (2024, April). Fuzzy logic integration in smart environmental management systems: Paving the way for a greener future. In International Conference on Smart Environment and Green Technologies(pp. 381–389). Cham: Springer Nature Switzerland.

 

 

Share your valuable work from Social Media Buttons