Learning single-view 3D reconstruction with limited pose supervision
Research output: Contribution to journal › Conference article › Research › peer-review
Standard
Learning single-view 3D reconstruction with limited pose supervision. / Yang, Guandao; Cui, Yin; Belongie, Serge; Hariharan, Bharath.
In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2018, p. 90-105.Research output: Contribution to journal › Conference article › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Learning single-view 3D reconstruction with limited pose supervision
AU - Yang, Guandao
AU - Cui, Yin
AU - Belongie, Serge
AU - Hariharan, Bharath
N1 - Publisher Copyright: © Springer Nature Switzerland AG 2018.
PY - 2018
Y1 - 2018
N2 - It is expensive to label images with 3D structure or precise camera pose. Yet, this is precisely the kind of annotation required to train single-view 3D reconstruction models. In contrast, unlabeled images or images with just category labels are easy to acquire, but few current models can use this weak supervision. We present a unified framework that can combine both types of supervision: a small amount of camera pose annotations are used to enforce pose-invariance and view-point consistency, and unlabeled images combined with an adversarial loss are used to enforce the realism of rendered, generated models. We use this unified framework to measure the impact of each form of supervision in three paradigms: semi-supervised, multi-task, and transfer learning. We show that with a combination of these ideas, we can train single-view reconstruction models that improve up to 7 points in performance (AP) when using only 1% pose annotated training data.
AB - It is expensive to label images with 3D structure or precise camera pose. Yet, this is precisely the kind of annotation required to train single-view 3D reconstruction models. In contrast, unlabeled images or images with just category labels are easy to acquire, but few current models can use this weak supervision. We present a unified framework that can combine both types of supervision: a small amount of camera pose annotations are used to enforce pose-invariance and view-point consistency, and unlabeled images combined with an adversarial loss are used to enforce the realism of rendered, generated models. We use this unified framework to measure the impact of each form of supervision in three paradigms: semi-supervised, multi-task, and transfer learning. We show that with a combination of these ideas, we can train single-view reconstruction models that improve up to 7 points in performance (AP) when using only 1% pose annotated training data.
KW - Few-shot learning
KW - GANs
KW - Single-image 3D-reconstruction
UR - http://www.scopus.com/inward/record.url?scp=85055416580&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-01267-0_6
DO - 10.1007/978-3-030-01267-0_6
M3 - Conference article
AN - SCOPUS:85055416580
SP - 90
EP - 105
JO - Lecture Notes in Computer Science
JF - Lecture Notes in Computer Science
SN - 0302-9743
T2 - 15th European Conference on Computer Vision, ECCV 2018
Y2 - 8 September 2018 through 14 September 2018
ER -
ID: 301825834