Neural weakly supervised fact check-worthiness detection with contrastive sampling-based ranking loss
Publikation: Bidrag til tidsskrift › Konferenceartikel › fagfællebedømt
Standard
Neural weakly supervised fact check-worthiness detection with contrastive sampling-based ranking loss. / Hansen, Casper; Hansen, Christian; Simonsen, Jakob Grue; Lioma, Christina.
I: CEUR Workshop Proceedings, Bind 2380, 2019.Publikation: Bidrag til tidsskrift › Konferenceartikel › fagfællebedømt
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Neural weakly supervised fact check-worthiness detection with contrastive sampling-based ranking loss
AU - Hansen, Casper
AU - Hansen, Christian
AU - Simonsen, Jakob Grue
AU - Lioma, Christina
PY - 2019
Y1 - 2019
N2 - This paper describes the winning approach used by the Copenhagen team in the CLEF-2019 CheckThat! lab. Given a political debate or speech, the aim is to predict which sentences should be prioritized for fact-checking by creating a ranked list of sentences. While many approaches for check-worthiness exist, we are the first to directly optimize the sentence ranking as all previous work has solely used standard classification based loss functions. We present a recurrent neural network model that learns a sentence encoding, from which a check-worthiness score is predicted. The model is trained by jointly optimizing a binary cross entropy loss, as well as a ranking based pairwise hinge loss. We obtain sentence pairs for training through contrastive sampling, where for each sentence we find the k most semantically similar sentences with opposite label. To increase the generalizability of the model, we utilize weak supervision by using an existing check-worthiness approach to weakly label a large unlabeled dataset. We experimentally show that both weak supervision and the ranking component improve the results individually (MAP increases of 25% and 9% respectively), while when used together improve the results even more (39% increase). Through a comparison to existing state-of-the-art check-worthiness methods, we find that our approach improves the MAP score by 11%.
AB - This paper describes the winning approach used by the Copenhagen team in the CLEF-2019 CheckThat! lab. Given a political debate or speech, the aim is to predict which sentences should be prioritized for fact-checking by creating a ranked list of sentences. While many approaches for check-worthiness exist, we are the first to directly optimize the sentence ranking as all previous work has solely used standard classification based loss functions. We present a recurrent neural network model that learns a sentence encoding, from which a check-worthiness score is predicted. The model is trained by jointly optimizing a binary cross entropy loss, as well as a ranking based pairwise hinge loss. We obtain sentence pairs for training through contrastive sampling, where for each sentence we find the k most semantically similar sentences with opposite label. To increase the generalizability of the model, we utilize weak supervision by using an existing check-worthiness approach to weakly label a large unlabeled dataset. We experimentally show that both weak supervision and the ranking component improve the results individually (MAP increases of 25% and 9% respectively), while when used together improve the results even more (39% increase). Through a comparison to existing state-of-the-art check-worthiness methods, we find that our approach improves the MAP score by 11%.
KW - Contrastive ranking
KW - Fact check-worthiness
KW - Neural networks
M3 - Conference article
AN - SCOPUS:85070534030
VL - 2380
JO - CEUR Workshop Proceedings
JF - CEUR Workshop Proceedings
SN - 1613-0073
T2 - 20th Working Notes of CLEF Conference and Labs of the Evaluation Forum, CLEF 2019
Y2 - 9 September 2019 through 12 September 2019
ER -
ID: 227228125