Loading...
Automated prediction of examinee proficiency from short-answer questions
Ha, Le Yaneva, Victoria Harik, Polina Pandian, Ravi Morales, Amy Clauser, Brian
Ha, Le
Yaneva, Victoria
Harik, Polina
Pandian, Ravi
Morales, Amy
Clauser, Brian
Editors
Other contributors
Affiliation
Epub Date
Issue Date
2020-12-10
Submitted date
Alternative
Abstract
This paper brings together approaches from the fields of NLP and psychometric measurement to address the problem of predicting examinee proficiency from responses to short-answer questions (SAQs). While previous approaches train on manually labeled data to predict the human ratings assigned to SAQ responses, the approach presented here models examinee proficiency directly and does not require manually labeled data to train on. We use data from a large medical exam where experimental SAQ items are embedded alongside 106 scored multiple-choice questions (MCQs). First, the latent trait of examinee proficiency is measured using the scored MCQs and then a model is trained on the experimental SAQ responses as input, aiming to predict proficiency as its target variable. The predicted value is then used as a “score” for the SAQ response and evaluated in terms of its contribution to the precision of proficiency estimation.
Citation
Ha, L.A., Yaneva, V., Harik, P., Pandian, R., Morales, A. and Clauser, B. (2020) Automated prediction of examinee proficiency from short-answer questions, Proceedings of the 28th International Conference on Computational Linguistics, pages 893–903 Barcelona, Spain (Online), December 8-13, 2020.
Journal
Research Unit
DOI
PubMed ID
PubMed Central ID
Embedded videos
Additional Links
Type
Conference contribution
Language
en
Description
© 2020 The Authors. Published by International Committee on Computational Linguistics. This is an open access article available under a Creative Commons licence.
The published version can be accessed at the following link on the publisher’s website: https://www.aclweb.org/anthology/2020.coling-main.77/
Series/Report no.
ISSN
EISSN
ISBN
9781952148279