Towards automatic generation of relevance judgments for a test collection
Abstract
This paper represents a new technique for building a relevance judgment list for information retrieval test collections without any human intervention. It is based on the number of occurrences of the documents in runs retrieved from several information retrieval systems and a distance based measure between the documents. The effectiveness of the technique is evaluated by computing the correlation between the ranking of the TREC systems using the original relevance judgment list (qrels) built by human assessors and the ranking obtained by using the newly generated qrels.Citation
Makary, M., Oakes, M., Yamout, F. (2016) 'Towards automatic generation of relevance judgments for a test collection', Eleventh International Conference on Digital Information Management, ICDIM 2016, September 19-21, Porto, Portugal: IEEE, pp. 121-126Publisher
IEEEJournal
Eleventh International Conference on Digital Information Management (ICDIM)Additional Links
http://ieeexplore.ieee.org/document/7829763/Type
Conference contributionLanguage
enISBN
9781509026425ae974a485f413a2113503eed53cd6c53
10.1109/ICDIM.2016.7829763
Scopus Count
Collections
The following licence applies to the copyright and re-use of this item:
- Creative Commons
Except where otherwise noted, this item's license is described as https://creativecommons.org/CC BY-NC-ND 4.0