Automated prediction of examinee proficiency from short-answer questions
MetadataShow full item record
AbstractThis paper brings together approaches from the fields of NLP and psychometric measurement to address the problem of predicting examinee proficiency from responses to short-answer questions (SAQs). While previous approaches train on manually labeled data to predict the human ratings assigned to SAQ responses, the approach presented here models examinee proficiency directly and does not require manually labeled data to train on. We use data from a large medical exam where experimental SAQ items are embedded alongside 106 scored multiple-choice questions (MCQs). First, the latent trait of examinee proficiency is measured using the scored MCQs and then a model is trained on the experimental SAQ responses as input, aiming to predict proficiency as its target variable. The predicted value is then used as a “score” for the SAQ response and evaluated in terms of its contribution to the precision of proficiency estimation.
CitationHa, L.A., Yaneva, V., Harik, P., Pandian, R., Morales, A. and Clauser, B. (2020) Automated prediction of examinee proficiency from short-answer questions, Proceedings of the 28th International Conference on Computational Linguistics, pages 893–903 Barcelona, Spain (Online), December 8-13, 2020.
Description© 2020 The Authors. Published by International Committee on Computational Linguistics. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://www.aclweb.org/anthology/2020.coling-main.77/
Except where otherwise noted, this item's license is described as https://creativecommons.org/licenses/by/4.0/