University of Glasgow at WebCLEF 2005: Experiments in per-field normalisation and language specific stemming
Research output: Chapter in Book/Report/Conference proceeding › Book chapter › Research › peer-review
Standard
University of Glasgow at WebCLEF 2005 : Experiments in per-field normalisation and language specific stemming. / Macdonald, C.; Plachouras, V.; He, B.; Lioma, Christina; Ounis, I.
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4022 LNCS 2006. p. 898-907.Research output: Chapter in Book/Report/Conference proceeding › Book chapter › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - CHAP
T1 - University of Glasgow at WebCLEF 2005
T2 - Experiments in per-field normalisation and language specific stemming
AU - Macdonald, C.
AU - Plachouras, V.
AU - He, B.
AU - Lioma, Christina
AU - Ounis, I.
PY - 2006/1/1
Y1 - 2006/1/1
N2 - We participated in the WebCLEF 2005 monolingual task. In this task, a search system aims to retrieve relevant documents from a multilingual corpus of Web documents from Web sites of European governments. Both the documents and the queries are written in a wide range of European languages. A challenge in this setting is to detect the language of documents and topics, and to process them appropriately. We develop a language specific technique for applying the correct stemming approach, as well as for removing the correct stopwords from the queries. We represent documents using three fields, namely content, title, and anchor text of incoming hyperlinks. We use a technique called per-field normalisation, which extends the Divergence From Randomness (DFR) framework, to normalise the term frequencies, and to combine them across the three fields. We also employ the length of the URL path of Web documents. The ranking is based on combinations of both the language specific stemming, if applied, and the per-field normalisation. We use our Terrier platform for all our experiments. The overall performance of our techniques is outstanding, achieving the overall top four performing runs, as well as the top performing run without metadata in the monolingual task. The best run only uses per-field normalisation, without applying stemming.
AB - We participated in the WebCLEF 2005 monolingual task. In this task, a search system aims to retrieve relevant documents from a multilingual corpus of Web documents from Web sites of European governments. Both the documents and the queries are written in a wide range of European languages. A challenge in this setting is to detect the language of documents and topics, and to process them appropriately. We develop a language specific technique for applying the correct stemming approach, as well as for removing the correct stopwords from the queries. We represent documents using three fields, namely content, title, and anchor text of incoming hyperlinks. We use a technique called per-field normalisation, which extends the Divergence From Randomness (DFR) framework, to normalise the term frequencies, and to combine them across the three fields. We also employ the length of the URL path of Web documents. The ranking is based on combinations of both the language specific stemming, if applied, and the per-field normalisation. We use our Terrier platform for all our experiments. The overall performance of our techniques is outstanding, achieving the overall top four performing runs, as well as the top performing run without metadata in the monolingual task. The best run only uses per-field normalisation, without applying stemming.
UR - http://www.scopus.com/inward/record.url?scp=33749608768&partnerID=8YFLogxK
M3 - Book chapter
AN - SCOPUS:33749608768
SN - 9783540456971
VL - 4022 LNCS
SP - 898
EP - 907
BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ER -
ID: 49502112