repro_eval: A Python Interface to Reproducibility Measures of System-Oriented IR Experiments

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review


  • Fulltext

    Submitted manuscript, 426 KB, PDF document

In this work we introduce repro_eval - a tool for reactive reproducibility studies of system-oriented Information Retrieval (IR) experiments. The corresponding Python package provides IR researchers with measures for different levels of reproduction when evaluating their systems’ outputs. By offering an easily extensible interface, we hope to stimulate common practices when conducting a reproducibility study of system-oriented IR experiments.

Original languageEnglish
Title of host publicationAdvances in Information Retrieval - 43rd European Conference on IR Research, ECIR 2021, Proceedings
EditorsDjoerd Hiemstra, Marie-Francine Moens, Josiane Mothe, Raffaele Perego, Martin Potthast, Fabrizio Sebastiani
Number of pages6
Publication date2021
ISBN (Print)9783030722395
Publication statusPublished - 2021
Event43rd European Conference on Information Retrieval, ECIR 2021 - Virtual, Online
Duration: 28 Mar 20211 Apr 2021


Conference43rd European Conference on Information Retrieval, ECIR 2021
ByVirtual, Online
SeriesLecture Notes in Computer Science
Volume12657 LNCS

Bibliographical note

Publisher Copyright:
© 2021, Springer Nature Switzerland AG.

    Research areas

  • Evaluation, Replicability, Reproducibility

ID: 306679515