Quantitative Comparison of Monte-Carlo Dropout Uncertainty Measures for Multi-class Segmentation

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Standard

Quantitative Comparison of Monte-Carlo Dropout Uncertainty Measures for Multi-class Segmentation. / Camarasa, Robin; Bos, Daniel; Hendrikse, Jeroen; Nederkoorn, Paul; Kooi, Eline; van der Lugt, Aad; de Bruijne, Marleen.

Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis - 2nd International Workshop, UNSURE 2020, and 3rd International Workshop, GRAIL 2020, Held in Conjunction with MICCAI 2020, Proceedings. ed. / Carole H. Sudre; Hamid Fehri; Tal Arbel; Christian F. Baumgartner; Adrian Dalca; Ryutaro Tanno; Koen Van Leemput; William M. Wells; Aristeidis Sotiras; Bartlomiej Papiez; Enzo Ferrante; Sarah Parisot. Springer, 2020. p. 32-41 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 12443 LNCS).

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Harvard

Camarasa, R, Bos, D, Hendrikse, J, Nederkoorn, P, Kooi, E, van der Lugt, A & de Bruijne, M 2020, Quantitative Comparison of Monte-Carlo Dropout Uncertainty Measures for Multi-class Segmentation. in CH Sudre, H Fehri, T Arbel, CF Baumgartner, A Dalca, R Tanno, K Van Leemput, WM Wells, A Sotiras, B Papiez, E Ferrante & S Parisot (eds), Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis - 2nd International Workshop, UNSURE 2020, and 3rd International Workshop, GRAIL 2020, Held in Conjunction with MICCAI 2020, Proceedings. Springer, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12443 LNCS, pp. 32-41, 2nd International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2020, and the 3rd International Workshop on Graphs in Biomedical Image Analysis, GRAIL 2020, held in conjunction with the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020, Lima, Peru, 08/10/2020. https://doi.org/10.1007/978-3-030-60365-6_4

APA

Camarasa, R., Bos, D., Hendrikse, J., Nederkoorn, P., Kooi, E., van der Lugt, A., & de Bruijne, M. (2020). Quantitative Comparison of Monte-Carlo Dropout Uncertainty Measures for Multi-class Segmentation. In C. H. Sudre, H. Fehri, T. Arbel, C. F. Baumgartner, A. Dalca, R. Tanno, K. Van Leemput, W. M. Wells, A. Sotiras, B. Papiez, E. Ferrante, & S. Parisot (Eds.), Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis - 2nd International Workshop, UNSURE 2020, and 3rd International Workshop, GRAIL 2020, Held in Conjunction with MICCAI 2020, Proceedings (pp. 32-41). Springer. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) Vol. 12443 LNCS https://doi.org/10.1007/978-3-030-60365-6_4

Vancouver

Camarasa R, Bos D, Hendrikse J, Nederkoorn P, Kooi E, van der Lugt A et al. Quantitative Comparison of Monte-Carlo Dropout Uncertainty Measures for Multi-class Segmentation. In Sudre CH, Fehri H, Arbel T, Baumgartner CF, Dalca A, Tanno R, Van Leemput K, Wells WM, Sotiras A, Papiez B, Ferrante E, Parisot S, editors, Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis - 2nd International Workshop, UNSURE 2020, and 3rd International Workshop, GRAIL 2020, Held in Conjunction with MICCAI 2020, Proceedings. Springer. 2020. p. 32-41. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 12443 LNCS). https://doi.org/10.1007/978-3-030-60365-6_4

Author

Camarasa, Robin ; Bos, Daniel ; Hendrikse, Jeroen ; Nederkoorn, Paul ; Kooi, Eline ; van der Lugt, Aad ; de Bruijne, Marleen. / Quantitative Comparison of Monte-Carlo Dropout Uncertainty Measures for Multi-class Segmentation. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis - 2nd International Workshop, UNSURE 2020, and 3rd International Workshop, GRAIL 2020, Held in Conjunction with MICCAI 2020, Proceedings. editor / Carole H. Sudre ; Hamid Fehri ; Tal Arbel ; Christian F. Baumgartner ; Adrian Dalca ; Ryutaro Tanno ; Koen Van Leemput ; William M. Wells ; Aristeidis Sotiras ; Bartlomiej Papiez ; Enzo Ferrante ; Sarah Parisot. Springer, 2020. pp. 32-41 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 12443 LNCS).

Bibtex

@inproceedings{61b6b99ee3e340478c6108e11e4fead2,
title = "Quantitative Comparison of Monte-Carlo Dropout Uncertainty Measures for Multi-class Segmentation",
abstract = "Over the past decade, deep learning has become the gold standard for automatic medical image segmentation. Every segmentation task has an underlying uncertainty due to image resolution, annotation protocol, etc. Therefore, a number of methods and metrics have been proposed to quantify the uncertainty of neural networks mostly based on Bayesian deep learning, ensemble learning methods or output probability calibration. The aim of our research is to assess how reliable the different uncertainty metrics found in the literature are. We propose a quantitative and statistical comparison of uncertainty measures based on the relevance of the uncertainty map to predict misclassification. Four uncertainty metrics were compared over a set of 144 models. The application studied is the segmentation of the lumen and vessel wall of carotid arteries based on multiple sequences of magnetic resonance (MR) images in multi-center data.",
author = "Robin Camarasa and Daniel Bos and Jeroen Hendrikse and Paul Nederkoorn and Eline Kooi and {van der Lugt}, Aad and {de Bruijne}, Marleen",
year = "2020",
doi = "10.1007/978-3-030-60365-6_4",
language = "English",
isbn = "9783030603649",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer",
pages = "32--41",
editor = "Sudre, {Carole H.} and Hamid Fehri and Tal Arbel and Baumgartner, {Christian F.} and Adrian Dalca and Ryutaro Tanno and {Van Leemput}, Koen and Wells, {William M.} and Aristeidis Sotiras and Bartlomiej Papiez and Enzo Ferrante and Sarah Parisot",
booktitle = "Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis - 2nd International Workshop, UNSURE 2020, and 3rd International Workshop, GRAIL 2020, Held in Conjunction with MICCAI 2020, Proceedings",
address = "Switzerland",
note = "2nd International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2020, and the 3rd International Workshop on Graphs in Biomedical Image Analysis, GRAIL 2020, held in conjunction with the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020 ; Conference date: 08-10-2020 Through 08-10-2020",

}

RIS

TY - GEN

T1 - Quantitative Comparison of Monte-Carlo Dropout Uncertainty Measures for Multi-class Segmentation

AU - Camarasa, Robin

AU - Bos, Daniel

AU - Hendrikse, Jeroen

AU - Nederkoorn, Paul

AU - Kooi, Eline

AU - van der Lugt, Aad

AU - de Bruijne, Marleen

PY - 2020

Y1 - 2020

N2 - Over the past decade, deep learning has become the gold standard for automatic medical image segmentation. Every segmentation task has an underlying uncertainty due to image resolution, annotation protocol, etc. Therefore, a number of methods and metrics have been proposed to quantify the uncertainty of neural networks mostly based on Bayesian deep learning, ensemble learning methods or output probability calibration. The aim of our research is to assess how reliable the different uncertainty metrics found in the literature are. We propose a quantitative and statistical comparison of uncertainty measures based on the relevance of the uncertainty map to predict misclassification. Four uncertainty metrics were compared over a set of 144 models. The application studied is the segmentation of the lumen and vessel wall of carotid arteries based on multiple sequences of magnetic resonance (MR) images in multi-center data.

AB - Over the past decade, deep learning has become the gold standard for automatic medical image segmentation. Every segmentation task has an underlying uncertainty due to image resolution, annotation protocol, etc. Therefore, a number of methods and metrics have been proposed to quantify the uncertainty of neural networks mostly based on Bayesian deep learning, ensemble learning methods or output probability calibration. The aim of our research is to assess how reliable the different uncertainty metrics found in the literature are. We propose a quantitative and statistical comparison of uncertainty measures based on the relevance of the uncertainty map to predict misclassification. Four uncertainty metrics were compared over a set of 144 models. The application studied is the segmentation of the lumen and vessel wall of carotid arteries based on multiple sequences of magnetic resonance (MR) images in multi-center data.

UR - http://www.scopus.com/inward/record.url?scp=85093121226&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-60365-6_4

DO - 10.1007/978-3-030-60365-6_4

M3 - Article in proceedings

AN - SCOPUS:85093121226

SN - 9783030603649

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 32

EP - 41

BT - Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis - 2nd International Workshop, UNSURE 2020, and 3rd International Workshop, GRAIL 2020, Held in Conjunction with MICCAI 2020, Proceedings

A2 - Sudre, Carole H.

A2 - Fehri, Hamid

A2 - Arbel, Tal

A2 - Baumgartner, Christian F.

A2 - Dalca, Adrian

A2 - Tanno, Ryutaro

A2 - Van Leemput, Koen

A2 - Wells, William M.

A2 - Sotiras, Aristeidis

A2 - Papiez, Bartlomiej

A2 - Ferrante, Enzo

A2 - Parisot, Sarah

PB - Springer

T2 - 2nd International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2020, and the 3rd International Workshop on Graphs in Biomedical Image Analysis, GRAIL 2020, held in conjunction with the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020

Y2 - 8 October 2020 through 8 October 2020

ER -

ID: 250446141