Neural Puppet: Generative Layered Cartoon Characters
Publikation: Bidrag til tidsskrift › Konferenceartikel › Forskning › fagfællebedømt
We propose a learning based method for generating new animations of a cartoon character given a few example images. Our method is designed to learn from a traditionally animated sequence, where each frame is drawn by an artist, and thus the input images lack any common structure, correspondences, or labels. We express pose changes as a deformation of a layered 2.5D template mesh, and devise a novel architecture that learns to predict mesh deformations matching the template to a target image. This enables us to extract a common low-dimensional structure from a diverse set of character poses. We combine recent advances in differentiable rendering as well as mesh-aware models to successfully align common template even if only a few character images are available during training. In addition to coarse poses, character appearance also varies due to shading, out-of-plane motions, and artistic effects. We capture these subtle changes by applying an image translation network to refine the mesh rendering, providing an end-to-end model to generate new animations of a character with high visual quality. We demonstrate that our generative model can be used to synthesize in-between frames and to create data-driven deformation. Our template fitting procedure outperforms state-of-the-art generic techniques for detecting image correspondences.
Originalsprog | Engelsk |
---|---|
Tidsskrift | Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020 |
Sider (fra-til) | 3335-3345 |
Antal sider | 11 |
DOI | |
Status | Udgivet - mar. 2020 |
Eksternt udgivet | Ja |
Begivenhed | 2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020 - Snowmass Village, USA Varighed: 1 mar. 2020 → 5 mar. 2020 |
Konference
Konference | 2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020 |
---|---|
Land | USA |
By | Snowmass Village |
Periode | 01/03/2020 → 05/03/2020 |
Sponsor | CVF, IEEE Computer Society |
Bibliografisk note
Publisher Copyright:
© 2020 IEEE.
ID: 301823072