Header logo is

Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimation


Conference Paper


Direct prediction of 3D body pose and shape remains a challenge even for highly parameterized deep learning models. Mapping from the 2D image space to the prediction space is difficult: perspective ambiguities make the loss function noisy and training data is scarce. In this paper, we propose a novel approach (Neural Body Fitting (NBF)). It integrates a statistical body model within a CNN, leveraging reliable bottom-up semantic body part segmentation and robust top-down body model constraints. NBF is fully differentiable and can be trained using 2D and 3D annotations. In detailed experiments, we analyze how the components of our model affect performance, especially the use of part segmentations as an explicit intermediate representation, and present a robust, efficiently trainable framework for 3D human pose estimation from 2D images with competitive results on standard benchmarks. Code is available at https://github.com/mohomran/neural_body_fitting

Award: (Best Student Paper Award)
Author(s): Mohamed Omran and Christoph Lassner and Gerard Pons-Moll and Peter V. Gehler and Bernt Schiele
Book Title: 3DV
Year: 2018
Month: September

Department(s): Perceiving Systems
Research Project(s): 3D Body Shape and Pose from Images
Bibtex Type: Conference Paper (conference)
Paper Type: Conference

Award Paper: Best Student Paper Award

Links: arXiv


  title = {Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimation},
  author = {Omran, Mohamed and Lassner, Christoph and Pons-Moll, Gerard and Gehler, Peter V. and Schiele, Bernt},
  booktitle = {3DV},
  month = sep,
  year = {2018},
  month_numeric = {9}