Fact Check-Worthiness Detection with Contrastive Ranking

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Check-worthiness detection aims at predicting which sentences should be prioritized for fact-checking. A typical use is to rank sentences in political debates and speeches according to their degree of check-worthiness. We present the first direct optimization of sentence ranking for check-worthiness; in contrast, all previous work has solely used standard classification based loss functions. We present a recurrent neural network model that learns a sentence encoding, from which a check-worthiness score is predicted. The model is trained by jointly optimizing a binary cross entropy loss, as well as a ranking based pairwise hinge loss. We obtain sentence pairs for training through contrastive sampling, where for each sentence we find the top most semantically similar sentences with opposite label. Through a comparison to existing state-of-the-art check-worthiness methods, we find that our approach improves the MAP score by 11%.

Original languageEnglish
Title of host publicationExperimental IR Meets Multilinguality, Multimodality, and Interaction - 11th International Conference of the CLEF Association, CLEF 2020, Proceedings
EditorsAvi Arampatzis, Evangelos Kanoulas, Theodora Tsikrika, Stefanos Vrochidis, Hideo Joho, Christina Lioma, Carsten Eickhoff, Aurélie Névéol, Aurélie Névéol, Linda Cappellato, Nicola Ferro
PublisherSpringer
Publication date2020
Pages124-130
ISBN (Print)9783030582180
DOIs
Publication statusPublished - 2020
Event11th Conference and Labs of the Evaluation Forum, CLEF 2020 - Thessaloniki, Greece
Duration: 22 Sep 202025 Sep 2020

Conference

Conference11th Conference and Labs of the Evaluation Forum, CLEF 2020
LandGreece
ByThessaloniki
Periode22/09/202025/09/2020
SeriesLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12260 LNCS
ISSN0302-9743

    Research areas

  • Check-worthiness, Contrastive ranking, Neural networks

ID: 250486804