Rethinking how to extend average precision to graded relevance
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
We present two new measures of retrieval effectiveness, inspired by Graded Average Precision(GAP), which extends Average Precision(AP) to graded relevance judgements. Starting from the random choice of a user, we define Extended Graded Average Precision(xGAP) and Expected Graded Average Precision(eGAP), which are more accurate than GAP in the case of a small number of highly relevant documents with high probability to be considered relevant by the users. The proposed measures are then evaluated on TREC 10, TREC 14, and TREC 21 collections showing that they actually grasp a different angle from GAP and that they are robust when it comes to incomplete judgments and shallow pools.
Original language | English |
---|---|
Title of host publication | Information Access Evaluation : Multilinguality, Multimodality, and Interaction - 5th International Conference of the CLEF Initiative, CLEF 2014, Proceedings |
Number of pages | 12 |
Publisher | Springer Verlag, |
Publication date | 1 Jan 2014 |
Pages | 19-30 |
ISBN (Print) | 9783319113814 |
DOIs | |
Publication status | Published - 1 Jan 2014 |
Externally published | Yes |
Event | 5th International Conference of the CLEF Initiative, CLEF 2014 - Sheffield, United Kingdom Duration: 15 Sep 2014 → 18 Sep 2014 |
Conference
Conference | 5th International Conference of the CLEF Initiative, CLEF 2014 |
---|---|
Land | United Kingdom |
By | Sheffield |
Periode | 15/09/2014 → 18/09/2014 |
Series | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|
Volume | 8685 LNCS |
ISSN | 0302-9743 |
ID: 216517815