Show simple item record

dc.contributor.authorHa, Le
dc.contributor.authorYaneva, Victoria
dc.contributor.authorBalwin, Peter
dc.contributor.authorMee, Janet
dc.date.accessioned2019-08-15T10:36:01Z
dc.date.available2019-08-15T10:36:01Z
dc.date.issued2019-08-02
dc.identifier.citationHa, L. A., Yaneva, V., Baldwin, P. and Mee, J. (2019) Predicting the difficulty of multiple choice questions in a high-stakes medical exam, Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications. Florence, Italy: Association for Computational Linguistics, pp. 11–20.en
dc.identifier.isbn9781950737345
dc.identifier.urihttp://hdl.handle.net/2436/622649
dc.description.abstractPredicting the construct-relevant difficulty of Multiple-Choice Questions (MCQs) has the potential to reduce cost while maintaining the quality of high-stakes exams. In this paper, we propose a method for estimating the difficulty of MCQs from a high-stakes medical exam, where all questions were deliberately written to a common reading level. To accomplish this, we extract a large number of linguistic features and embedding types, as well as features quantifying the difficulty of the items for an automatic question-answering system. The results show that the proposed approach outperforms various baselines with a statistically significant difference. Best results were achieved when using the full feature set, where embeddings had the highest predictive power, followed by linguistic features. An ablation study of the various types of linguistic features suggested that information from all levels of linguistic processing contributes to predicting item difficulty, with features related to semantic ambiguity and the psycholinguistic properties of words having a slightly higher importance. Owing to its generic nature, the presented approach has the potential to generalize over other exams containing MCQs.en
dc.formatapplication/PDFen
dc.language.isoenen
dc.publisherAssociation for Computational Linguisticsen
dc.relation.urlhttps://aclweb.org/anthology/papers/W/W19/W19-4402/en
dc.subjectItem difficultyen
dc.titlePredicting the difficulty of multiple choice questions in a high-stakes medical examen
dc.typeConference contributionen
dc.date.updated2019-08-14T16:59:32Z
dc.conference.nameFourteenth Workshop on Innovative Use of NLP for Building Educational Applications
dc.date.accepted2019-05-24
rioxxterms.funderUniversity of Wolverhamptonen
rioxxterms.identifier.projectUOW150819LHen
rioxxterms.versionAMen
rioxxterms.licenseref.urihttps://creativecommons.org/licenses/by/4.0/en
rioxxterms.licenseref.startdate2019-08-15en
refterms.dateFCD2019-08-15T10:35:41Z
refterms.versionFCDAM
refterms.dateFOA2019-08-15T10:36:01Z


Files in this item

Thumbnail
Name:
BEA_2019_Predicting_P_value ...
Size:
179.8Kb
Format:
PDF

This item appears in the following Collection(s)

Show simple item record