TY - JOUR

T1 - Differentiable Depth for Real2Sim Calibration of Soft Body Simulations

AU - Arnavaz, K.

AU - Nielsen, M. Kragballe

AU - Kry, P. G.

AU - Macklin, M.

AU - Erleben, K.

N1 - Publisher Copyright: © 2022 The Authors. Computer Graphics Forum published by Eurographics - The European Association for Computer Graphics and John Wiley & Sons Ltd.

PY - 2023

Y1 - 2023

N2 - In this work, we present a novel approach for calibrating material model parameters for soft body simulations using real data. We use a fully differentiable pipeline, combining a differentiable soft body simulator and differentiable depth rendering, which permits fast gradient-based optimizations. Our method requires no data pre-processing, and minimal experimental set-up, as we directly minimize the L2-norm between raw LIDAR scans and rendered simulation states. In essence, we provide the first marker-free approach for calibrating a soft-body simulator to match observed real-world deformations. Our approach is inexpensive as it solely requires a consumer-level LIDAR sensor compared to acquiring a professional marker-based motion capture system. We investigate the effects of different material parameterizations and evaluate convergence for parameter optimization in both single and multi-material scenarios of varying complexity. Finally, we show that our set-up can be extended to optimize for dynamic behaviour as well.

AB - In this work, we present a novel approach for calibrating material model parameters for soft body simulations using real data. We use a fully differentiable pipeline, combining a differentiable soft body simulator and differentiable depth rendering, which permits fast gradient-based optimizations. Our method requires no data pre-processing, and minimal experimental set-up, as we directly minimize the L2-norm between raw LIDAR scans and rendered simulation states. In essence, we provide the first marker-free approach for calibrating a soft-body simulator to match observed real-world deformations. Our approach is inexpensive as it solely requires a consumer-level LIDAR sensor compared to acquiring a professional marker-based motion capture system. We investigate the effects of different material parameterizations and evaluate convergence for parameter optimization in both single and multi-material scenarios of varying complexity. Finally, we show that our set-up can be extended to optimize for dynamic behaviour as well.

KW - animation

KW - methods and applications

KW - physically based animation

KW - ray tracing

KW - rendering

KW - robotics

U2 - 10.1111/cgf.14720

DO - 10.1111/cgf.14720

M3 - Journal article

AN - SCOPUS:85143054771

VL - 42

SP - 277

EP - 289

JO - Computer Graphics Forum (Print)

JF - Computer Graphics Forum (Print)

SN - 0167-7055

IS - 1

ER -