TY - JOUR

T1 - Semantic similarity metrics for image registration

AU - Czolbe, Steffen

AU - Pegios, Paraskevas

AU - Krause, Oswin

AU - Feragen, Aasa

N1 - Publisher Copyright: © 2023 The Author(s)

PY - 2023

Y1 - 2023

N2 - Image registration aims to find geometric transformations that align images. Most algorithmic and deep learning-based methods solve the registration problem by minimizing a loss function, consisting of a similarity metric comparing the aligned images, and a regularization term ensuring smoothness of the transformation. Existing similarity metrics like Euclidean Distance or Normalized Cross-Correlation focus on aligning pixel intensity values or correlations, giving difficulties with low intensity contrast, noise, and ambiguous matching. We propose a semantic similarity metric for image registration, focusing on aligning image areas based on semantic correspondence instead. Our approach learns dataset-specific features that drive the optimization of a learning-based registration model. We train both an unsupervised approach extracting features with an auto-encoder, and a semi-supervised approach using supplemental segmentation data. We validate the semantic similarity metric using both deep-learning-based and algorithmic image registration methods. Compared to existing methods across four different image modalities and applications, the method achieves consistently high registration accuracy and smooth transformation fields.

AB - Image registration aims to find geometric transformations that align images. Most algorithmic and deep learning-based methods solve the registration problem by minimizing a loss function, consisting of a similarity metric comparing the aligned images, and a regularization term ensuring smoothness of the transformation. Existing similarity metrics like Euclidean Distance or Normalized Cross-Correlation focus on aligning pixel intensity values or correlations, giving difficulties with low intensity contrast, noise, and ambiguous matching. We propose a semantic similarity metric for image registration, focusing on aligning image areas based on semantic correspondence instead. Our approach learns dataset-specific features that drive the optimization of a learning-based registration model. We train both an unsupervised approach extracting features with an auto-encoder, and a semi-supervised approach using supplemental segmentation data. We validate the semantic similarity metric using both deep-learning-based and algorithmic image registration methods. Compared to existing methods across four different image modalities and applications, the method achieves consistently high registration accuracy and smooth transformation fields.

KW - Deep learning

KW - Image registration

KW - Representation learning

UR - http://www.scopus.com/inward/record.url?scp=85159141451&partnerID=8YFLogxK

U2 - 10.1016/j.media.2023.102830

DO - 10.1016/j.media.2023.102830

M3 - Journal article

C2 - 37172390

AN - SCOPUS:85159141451

VL - 87

JO - Medical Image Analysis

JF - Medical Image Analysis

SN - 1361-8415

M1 - 102830

ER -