Shape from Video: Dense Shape, Texture, Motion and Lighting from Monocular Image Streams

Azeem Lakdawalla and Aaron Hertzmann

University of Toronto



Abstract: This paper presents a probabilistic framework for robust recovery of dense 3D shape, motion, texture and lighting from monocular image streams. We assume that the object is smooth, Lambertian, illuminated by one distant light source, and subject to smoothly-varying rigid motion. The problem is formulated as a MAP estimation problem in which all shape, motion, noise variances and outlier probabilities are estimated simultaneously. Estimation is performed using a multi-stage initialization process followed by a large-scale quasi-Newtonian optimization technique.


A. Lakdawalla and A. Hertzmann. Shape From Video: Dense Shape, Texture, Motion and Lighting from Monocular Image Streams. IEEE Workshop on Photometric Analysis for Computer Vision, in conjunction with ICCV '07, Rio de Janeiro, Brazil.


Results


Input SequenceRecovered Shape


Occluded SequenceUsing our robust methodWithout robustness