AbstractIn this paper, we investigate the effects of using subword information in representation learning. We argue that using syntactic subword units effects the quality of the word representations positively. We introduce a morpheme-based model and compare it against to word-based, character-based, and character n-gram level models. Our model takes a list of candidate segmentations of a word and learns the representation of the word based on different segmentations that are weighted by an attention mechanism. We performed experiments on Turkish as a morphologically rich language and English with a comparably poorer morphology. The results show that morpheme-based models are better at learning word representations of morphologically complex languages compared to character-based and character n-gram level models since the morphemes help to incorporate more syntactic knowledge in learning, that makes morpheme-based models better at syntactic tasks.
CitationÜstün, A., Kurfalı, M. and Can, B. (2018) Characters or morphemes: how to represent words? In, Proceedings of The Third Workshop on Representation Learning for NLP, Augenstein, I., Cao, K., He, H., Hill, F. et al. Stroudsburg, PA: Association for Computational Linguistics, pp. 144-153.
Description© 2018 The Authors. Published by Association for Computational Linguistics. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: http://dx.doi.org/10.18653/v1/W18-3019
SponsorsThis research was supported by TUBITAK (The Scientific and Technological Research Council of Turkey) grant number 115E464.
Except where otherwise noted, this item's license is described as http://creativecommons.org/licenses/by/4.0/