Modeling morpheme triplets with a three-level hierarchical Dirichlet process
Abstract
Morphemes are not independent units and attached to each other based on morphotactics. However, they are assumed to be independent from each other to cope with the complexity in most of the models in the literature. We introduce a language independent model for unsupervised morphological segmentation using hierarchical Dirichlet process (HDP). We model the morpheme dependencies in terms of morpheme trigrams in each word. Trigrams, bigrams and unigrams are modeled within a three-level HDP, where the trigram Dirichlet process (DP) uses the bigram DP and bigram DP uses unigram DP as the base distribution. The results show that modeling morpheme dependencies improve the F-measure noticeably in English, Turkish and Finnish.Citation
Kumyol, S. and Can, B. (2016) Modeling morpheme triplets with a three-level hierarchical Dirichlet process, 2016 International Conference on Asian Language Processing (IALP), 21-23 November, 2016, Tainan, Taiwan.Publisher
IEEEAdditional Links
https://ieeexplore.ieee.org/document/7876007Type
Conference contributionLanguage
enDescription
This is an accepted manuscript of an article published by IEEE in 2016 International Conference on Asian Language Processing (IALP) on 13/03/2017, available online: https://ieeexplore.ieee.org/document/7876007 The accepted version of the publication may differ from the final published version.ISSN
2159-1962EISSN
2159-1970ae974a485f413a2113503eed53cd6c53
10.1109/ialp.2016.7876007
Scopus Count
Except where otherwise noted, this item's license is described as https://creativecommons.org/licenses/by-nc-nd/4.0/