External lecture by Reinhard Koch

Fast Tracking of Deformable Objects in Depth and Color Video

Professor Reinhard Koch, Christian-Albrechts-University of Kiel

16 May 2012, 10.15-11.15 in Visionen, B-building

3D modeling and tracking of rigid objects from video is nowadays a well established topic. In the case of non-rigid scenes and deformable objects, there is still a research demand. The recent availability of fast range video systems, like Time-of-Flight depth cameras or the Kinect-sensor, allows to capture time-varying 3D scenes and the modeling and tracking of moving nonrigid objects in real-time. However, the space of possible deformations might be of very high dimension, and high-dimensional realtime tracking is a demanding task.

In this talk I will present our work on deformable object tracking, that combines a model-based analysis-synthesis approach with CMA-ES, a potent stochastic solver for non-convex high-dimensional minimisation problems. The object is modeled by a deformable textured surface mesh, that is rendered (synthesized) into the camera view and compared with the color and depth data of the Kinect. The deformation is controlled by NxM 3D NURBS control points that are fitted to the data with CMA-ES (analysis). The deformation dimension is D=3xNxM, typically of size D=50-200, still the problem can be solved in near-realtime. I will conclude with some application scenarios for deformable object tracking.