Predicting lexical complexity in English texts: the Complex 2.0 dataset
Abstract
Identifying words which may cause difficulty for a reader is an essential step in most lexical text simplification systems prior to lexical substitution and can also be used for assessing the readability of a text. This task is commonly referred to as complex word identification (CWI) and is often modelled as a supervised classification problem. For training such systems, annotated datasets in which words and sometimes multi-word expressions are labelled regarding complexity are required. In this paper we analyze previous work carried out in this task and investigate the properties of CWI datasets for English. We develop a protocol for the annotation of lexical complexity and use this to annotate a new dataset, CompLex 2.0. We present experiments using both new and old datasets to investigate the nature of lexical complexity. We found that a Likert-scale annotation protocol provides an objective setting that is superior for identifying the complexity of words compared to a binary annotation protocol. We release a new dataset using our new protocol to promote the task of Lexical Complexity Prediction.Citation
Shardlow, M., Evans, R. & Zampieri, M. (2022) Predicting lexical complexity in English texts: the Complex 2.0 dataset. Lang Resources & Evaluation. https://doi.org/10.1007/s10579-022-09588-2Publisher
SpringerJournal
Language Resources and EvaluationAdditional Links
https://link.springer.com/article/10.1007/s10579-022-09588-2Type
Journal articleLanguage
enDescription
© 2022 The Authors. Published by Springer. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://doi.org/10.1007/s10579-022-09588-2ISSN
1574-020Xae974a485f413a2113503eed53cd6c53
10.1007/s10579-022-09588-2
Scopus Count
Except where otherwise noted, this item's license is described as https://creativecommons.org/licenses/by/4.0/