Towards automatic generation of relevance judgments for a test collection

2.50
Hdl Handle:
http://hdl.handle.net/2436/620407
Title:
Towards automatic generation of relevance judgments for a test collection
Authors:
Makary, Mireille; Oakes, Michael; Yamout, Fadi
Abstract:
This paper represents a new technique for building a relevance judgment list for information retrieval test collections without any human intervention. It is based on the number of occurrences of the documents in runs retrieved from several information retrieval systems and a distance based measure between the documents. The effectiveness of the technique is evaluated by computing the correlation between the ranking of the TREC systems using the original relevance judgment list (qrels) built by human assessors and the ranking obtained by using the newly generated qrels.
Publisher:
IEEE
Journal:
Conference paper: Eleventh International Conference on Digital Information Management (ICDIM), 2016
Issue Date:
Sep-2016
URI:
http://hdl.handle.net/2436/620407
DOI:
10.1109/ICDIM.2016.7829763
Additional Links:
http://ieeexplore.ieee.org/document/7829763/
Type:
Meetings and Proceedings
Language:
en
ISBN:
9781509026418
Appears in Collections:
FOSS

Full metadata record

DC FieldValue Language
dc.contributor.authorMakary, Mireilleen
dc.contributor.authorOakes, Michaelen
dc.contributor.authorYamout, Fadien
dc.date.accessioned2017-03-09T10:12:22Z-
dc.date.available2017-03-09T10:12:22Z-
dc.date.issued2016-09-
dc.identifier.isbn9781509026418en
dc.identifier.doi10.1109/ICDIM.2016.7829763-
dc.identifier.urihttp://hdl.handle.net/2436/620407-
dc.description.abstractThis paper represents a new technique for building a relevance judgment list for information retrieval test collections without any human intervention. It is based on the number of occurrences of the documents in runs retrieved from several information retrieval systems and a distance based measure between the documents. The effectiveness of the technique is evaluated by computing the correlation between the ranking of the TREC systems using the original relevance judgment list (qrels) built by human assessors and the ranking obtained by using the newly generated qrels.en
dc.language.isoenen
dc.publisherIEEEen
dc.relation.urlhttp://ieeexplore.ieee.org/document/7829763/en
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.subjectInformation Retrievalen
dc.subjectTest Collectionsen
dc.subjectRelevance Judgementsen
dc.titleTowards automatic generation of relevance judgments for a test collectionen
dc.typeMeetings and Proceedingsen
dc.identifier.journalConference paper: Eleventh International Conference on Digital Information Management (ICDIM), 2016en
dc.date.accepted2016-08-
rioxxterms.funderInternalen
rioxxterms.identifier.projectUoW090317MOen
rioxxterms.versionAMen
rioxxterms.licenseref.urihttps://creativecommons.org/CC BY-NC-ND 4.0en
rioxxterms.licenseref.startdate2017-03-09en
This item is licensed under a Creative Commons License
Creative Commons
All Items in WIRE are protected by copyright, with all rights reserved, unless otherwise indicated.