Multimodal Unsupervised Image-to-Image Translation

Publikation: Bidrag til tidsskriftKonferenceartikelForskningfagfællebedømt

Unsupervised image-to-image translation is an important and challenging problem in computer vision. Given an image in the source domain, the goal is to learn the conditional distribution of corresponding images in the target domain, without seeing any examples of corresponding image pairs. While this conditional distribution is inherently multimodal, existing approaches make an overly simplified assumption, modeling it as a deterministic one-to-one mapping. As a result, they fail to generate diverse outputs from a given source domain image. To address this limitation, we propose a Multimodal Unsupervised Image-to-image (MUNIT) framework. We assume that the image representation can be decomposed into a content code that is domain-invariant, and a style code that captures domain-specific properties. To translate an image to another domain, we recombine its content code with a random style code sampled from the style space of the target domain. We analyze the proposed framework and establish several theoretical results. Extensive experiments with comparisons to state-of-the-art approaches further demonstrate the advantage of the proposed framework. Moreover, our framework allows users to control the style of translation outputs by providing an example style image. Code and pretrained models are available at https://github.com/nvlabs/MUNIT.

OriginalsprogEngelsk
TidsskriftLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Sider (fra-til)179-196
Antal sider18
ISSN0302-9743
DOI
StatusUdgivet - 2018
Eksternt udgivetJa
Begivenhed15th European Conference on Computer Vision, ECCV 2018 - Munich, Tyskland
Varighed: 8 sep. 201814 sep. 2018

Konference

Konference15th European Conference on Computer Vision, ECCV 2018
LandTyskland
ByMunich
Periode08/09/201814/09/2018

Bibliografisk note

Publisher Copyright:
© 2018, Springer Nature Switzerland AG.

ID: 301825957