Exploring Construct Validity and Construct Underrepresentation in CBT-Based Use of English among Diploma Students.

Author

Ali Mohammed Also, Rasheedah Suleiman, Abdullahi Kaigama

Abstract

 This study assesses construct validity and construct underrepresentation in Computer-Based Testing (CBT) for Use of English courses among diploma students. Whereas CBT improves evaluation scalability and grading efficacy, automated components mostly overstress low-level recollection via multiple-choice set of questions, unconsciously eliminating higher-order language skills like impulsive writing, practical competence, and discourse management. Applying a mixed-methods paradigm to analyse test items, students’ feedback and performance metrics, the study highlights how procedural limitations and constricting item strategy lead to construct underrepresentation. Finally, the findings assert for the integration of hybrid item formats and genuine communicative tasks into Computer-based models to affirm detailed, valid assessments of academic English proficiency.

Keywords

Construct Validity, Construct Underrepresentation, Computer-Based Testing, Diploma Students.

Full Text:

Download Paper PDF

References

Abate, T., & Mishore, E. (2024). Alignment analysis between teacher-made tests with the learning objectives in a selected school of central regional state of Ethiopia. Heliyon, 10(11

Adebola, O. O. (2022). Pre-Service Teachers Experiences during Teaching Practice in South Africa: Challenges and Solutions. Journal of Education, Teaching and Learning, 7(1):66-72 https://dx.doi.org/10.26737/jetl.v7i1.3146

Agtarap, M., Varier, D., & Constantinou, E. (2024). Evaluating teacher-made test items and assessment preparation in higher education. International Journal of Research and Innovation in Social Science, 8(4), 1531–1547.

Al-Mekhlafi, M. A. A., & Naji, I. M. (2013). Supervisors’ practices in improving student-teachers’ grammar teaching strategies during practicum: A comparison of student teachers’ and supervisors’ views. Wudpecker Journal of Educational Research, 2(1), 6–17. https://www.researchgate.net/publication/287331834

Alshurafat, H., Al Shbail, M. O., & Masadeh, W. M. (2024). Cognitive level alignment of accounting examination questions using Bloom’s Revised Taxonomy: A content analysis approach. Journal of Accounting Education, 68, Article 100912. https://doi.org/10.1016/j.jaccedu.2024.100912

Anderson, J. (2018). The role of initial teacher training courses in the professional development of experienced non-native speaker English language teachers. English Language Teacher Education and Development, 21.a. http://www.elted.net/uploads/7/3/1/6/7316005/-anderson_2018.pdf

Ariza-Quiñones, K. L., Hernández-Polo, L. D., Lesmes-Lesmes, K. J., & Molina-Ramírez, E. L. (2022). Preservice EFL teachers’ experiences in their first teaching practicum: A collaborative autoethnography. Profile Issues in Teachers’ Professional Development, 24(1),193–207. a. https://revistas.unal.edu.co/index.php/profile/article/view/91259

Bartolome, D. J. P., & Navalta, J. D. (2025). Misalignment in cognitive levels: A taxonomic analysis of Tables of Specification and mathematics examinations. Journal of Education, Learning, and Management, 2(2), 194–207. https://doi.org/10.69739/jelm.v2i2.1025

Benton, S.L & Ryalls, R. K. (2016). Challenging misconceptions about students’ ratings of instruction [IDEA Paper 55. https://www.ideaedu.org/idea_papers/challenging-misconceptions-about-student-ratings-of-instruction/

Benton, S.L & Young, S. (2018). Best practices in the evaluation of teaching. IDEA Paper #69. https://files.eric.ed.gov/fulltext/ED588352.pdf

Botaccio, L. A., Ortega, J. L. G., Rincón, A. N., & Fuentes, A. R. (2020). Evaluation for teachers and students in higher education. Sustainability, 12(10), 4078. https://doi.org/10.3390/su12104078

Bowen, G. A. (2023). Document analysis as a qualitative research method. Qualitative Research Journal, 23(1), 14–28. https://doi.org/10.3316/qrj0902027

Chapelle, C. A., & Lee, Y. W. (2021). Argument-based validity in language testing: Foundations and applications. Language Testing, 38(1), 45–68. https://doi.org/10.1177/0265532220925487

Febriyana, A., & Harjanto, I. (2023). Cognitive level analysis of English semester examination questions based on Revised Bloom’s Taxonomy. Journal of English Language Teaching and Linguistics, 8(2), 145–160.

Gupta, S., Grover, S., Menon, V., Indu, P. V., Vidhukumar, K., & Chacko, D. (2025). Item generation and establishing face and content validity of a rating scale: A primer. Indian Journal of Psychiatry, 67(8), 816–822. https://doi.org/10.4103/indianjpsychiatry_750_25

Khoy, M. A. (2025). Evaluating the cognitive complexity of higher education language assessment tasks: A content analysis using Bloom’s Revised Taxonomy. System, 128, Article 103510.

Leung, C. (2025). Test validation: Beyond arguments in communicative language assessment. Language Teaching Research Quarterly, 50(11), 1–18.

Menon, V., Grover, S., Gupta, S., Indu, P. V., Chacko, D., & Vidhukumar, K. (2025). A primer on reliability testing of a rating scale. Indian Journal of Psychiatry, 67(7), 725–729. https://doi.org/10.4103/indianjpsychiatry_584_25

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement (3rd ed., pp. 13–103). Macmillan Publishing Company.

OECD. (2023). Digital assessment: OECD Digital Education Outlook 2023. OECD Publishing. https://doi.org/10.1787/c74f03de-en

Pengelley, R., Yeboah, A., & Weirich, S. (2025). Computer-based testing in higher education: A phenomenological investigation into undergraduate students' perspectives through the Technology Acceptance Model. BMC Education, 25(1), 102–118.

Ravand, H. (2025). Measuring cognitive levels in high-stakes testing: Item characteristic curve and CDM analysis of university entrance examinations. Frontiers in Education, 10, Article 1644811. https://doi.org/10.3389/feduc.2025.1644811

Schneider, I. (2017). Can we trust measures of political trust? Assessing measurement equivalence in diverse regime types. Social Indicators Research, 133(3), 963–984.

Shou, Y., Sellbom, M., & Chen, H.-F. (2022). Fundamentals of measurement in clinical psychology. In G. J. G. Asmundson (Ed.), Comprehensive Clinical Psychology (2nd Edition) (pp. 13–35). Elsevier. https://doi.org/10.1016/B978-0-12-818697-8.00110-2 

Sireci, S. G. (1998). The construct of content validity. Social Indicators Research, 45, 83–117. https://doi.org/10.1023/a:1006985528729 

Sportel, B. E., Aardema, H., Boonstra, N., Arends, J., Rudd, B., Metz, M. J., Castelein, S., & Pijnenborg, G. H. M. (2023). Measuring recovery in participants with a schizophrenia spectrum disorder: Validation of the Individual Recovery Outcomes Counter (I.ROC). BMC Psychiatry, 23(1), 296–296. https://doi.org/10.1186/s12888-023-04763-3

Tsagari, D., & Bøhn, H. (2025). Preface to "Recent developments in language testing and assessment". Languages, 10(2), Article 22. https://doi.org/10.3390/languages10020022

Xie, Q. (2024). Construct representation and predictive validity of integrated writing tasks: A study on automated language assessments. Assessing Writing, 61(4), Article 100846. https://doi.org/10.1016/j.asw.2024.100846

 

 

Share your valuable work from Social Media Buttons