Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Study
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Standard
Automated Medical Coding on MIMIC-III and MIMIC-IV : A Critical Review and Replicability Study. / Edin, Joakim; Junge, Alexander; Havtorn, Jakob D.; Borgholt, Lasse; Maistro, Maria; Ruotsalo, Tuukka; Maaløe, Lars.
SIGIR 2023 - Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, Inc., 2023. p. 2572-2582.Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Automated Medical Coding on MIMIC-III and MIMIC-IV
T2 - 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023
AU - Edin, Joakim
AU - Junge, Alexander
AU - Havtorn, Jakob D.
AU - Borgholt, Lasse
AU - Maistro, Maria
AU - Ruotsalo, Tuukka
AU - Maaløe, Lars
N1 - Publisher Copyright: © 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
PY - 2023
Y1 - 2023
N2 - Medical coding is the task of assigning medical codes to clinical free-text documentation. Healthcare professionals manually assign such codes to track patient diagnoses and treatments. Automated medical coding can considerably alleviate this administrative burden. In this paper, we reproduce, compare, and analyze state-of-the-art automated medical coding machine learning models. We show that several models underperform due to weak configurations, poorly sampled train-test splits, and insufficient evaluation. In previous work, the macro F1 score has been calculated sub-optimally, and our correction doubles it. We contribute a revised model comparison using stratified sampling and identical experimental setups, including hyperparameters and decision boundary tuning. We analyze prediction errors to validate and falsify assumptions of previous works. The analysis confirms that all models struggle with rare codes, while long documents only have a negligible impact. Finally, we present the first comprehensive results on the newly released MIMIC-IV dataset using the reproduced models. We release our code, model parameters, and new MIMIC-III and MIMIC-IV training and evaluation pipelines to accommodate fair future comparisons.
AB - Medical coding is the task of assigning medical codes to clinical free-text documentation. Healthcare professionals manually assign such codes to track patient diagnoses and treatments. Automated medical coding can considerably alleviate this administrative burden. In this paper, we reproduce, compare, and analyze state-of-the-art automated medical coding machine learning models. We show that several models underperform due to weak configurations, poorly sampled train-test splits, and insufficient evaluation. In previous work, the macro F1 score has been calculated sub-optimally, and our correction doubles it. We contribute a revised model comparison using stratified sampling and identical experimental setups, including hyperparameters and decision boundary tuning. We analyze prediction errors to validate and falsify assumptions of previous works. The analysis confirms that all models struggle with rare codes, while long documents only have a negligible impact. Finally, we present the first comprehensive results on the newly released MIMIC-IV dataset using the reproduced models. We release our code, model parameters, and new MIMIC-III and MIMIC-IV training and evaluation pipelines to accommodate fair future comparisons.
KW - Automated Medical Coding
KW - MIMIC
KW - Reproducibility
U2 - 10.1145/3539618.3591918
DO - 10.1145/3539618.3591918
M3 - Article in proceedings
AN - SCOPUS:85168694311
SP - 2572
EP - 2582
BT - SIGIR 2023 - Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval
PB - Association for Computing Machinery, Inc.
Y2 - 23 July 2023 through 27 July 2023
ER -
ID: 383786342