Mathematical Image Analysis Group, Saarland University

Variational Optical Flow Estimation

Tutorial at ICCV 2009, Kyoto, Japan, September 2009

Thomas Brox, U.C.Berkeley
Andrés Bruhn, Saarland University

Motivation: Many computer vision tasks require point correspondences between images. While in some of these fields, sparse point correspondences are sufficient, many others, such as dynamical textures, motion segmentation, 3D motion analysis, superresolution video, and action recognition, benefit enormously from dense correspondence fields. Variational methods are particularly well suited to compute dense optical flow fields, as they combine all available data with prior assumptions in a sound optimization framework and even allow for subpixel accuracy.

Goal: This course will provide a comprehensible introduction into the theoretical background of recent state-of-the-art methods and shows the modelling options that will allow the participants to adapt the model to their specific needs. The course will also cover the numerics behind the models which are decisive for implementation. The focus will be on efficiently solving the very large linear or nonlinear systems with multigrid methods, which allow to compute dense flow fields in real-time.

Duration

3-4 hours

Preliminary Course Outline

0. Introduction

I. Variational Basics (Bruhn)

Model Assumptions
Minimzation via Euler-Lagrange
Basic Numerics (Jacobi, Gauss-Seidel)

II. Modeling Aspects (Brox)

Discontinuity-Preserving Smoothness Terms
Robust Data Terms
Image Features
- Color
- Gradient
- Structure Texture Decomposition
- Photometric Invariants
- HOG
Large Displacements
- Warping
- Region Matching and HOG flow

III. Efficient Numerics (Bruhn)

Linear Solvers
- Succesive Overrelaxation
- Linear Multigrid
Nonlinear Solvers
- Nested Fixed Point Iteration
- Nonlinear Multigrid (Full Approximation Scheme)
Implementations on Parallel Hardware

IV. Summary

Recent Publications

• T.Brox, C.Bregler, J.Malik
Large displacement optical flow.
IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR 2009).
(integrates landmarks from region matching to handle large displacements)

• H.Zimmer, A.Bruhn, J.Weickert, L.Valgaerts, A.Salgado, B.Rosenhahn, HP.Seidel
Complementary optic flow.
Int. Conf. on Energy Minimization Methods in Computer Vision and Pattern
Recognition (EMMCVPR 2009).
(new complementary regularizer, top rank result in the Middlebury benchmark)

Short Biographies

Thomas Brox received the Ph.D. degree in computer science from the Saarland University, Saarbrücken, Germany in 2005. During his studies, he spent three months as a visiting researcher at the INRIA Sophia- Antipolis, France. After his Ph.D., he joined the Computer Vision Group at the University of Bonn, Germany, as a postdoctoral researcher. From October 2007 to October 2008 he headed the Intelligent System Group at the University of Dresden, Germany, as a temporary faculty member. He is now a postdoctoral fellow in the Computer Vision Group of Jitendra Malik at U.C. Berkeley. The scientific work of Thomas Brox comprises about 60 papers in the fields of segmentation, motion estimation, and tracking. He served on the program commitees of ICCV, ECCV, and CVPR, and regularly reviews for all major computer vision journals. In 2004 he received the Longuet-Higgins Best Paper Award for his work on optical flow at the European Conference on Computer Vision.

Andrés Bruhn received a diploma degree in Computer Engineering from the University of Mannheim, Mannheim, Germany, in 2001 and a Ph.D. degree in Computer Science from the Saarland University, Saabrücken, Germany, in 2006. After working as a postdoctoral researcher from 2006 to 2007, he currently holds the position of an assistant professor in the Mathematical Image Analysis Group at Saarland University, Saarbrücken, Germany. His research interests include optical flow estimation, stereo reconstruction, diffusion filtering and fast numerical schemes for image processing and computer vision. Dr. Bruhn served on the programme committee of ICCV, ECCV, ACCV, and CVPR and regularly reviews for all major computer vision journals. His publications received several awards, among them the 2004 Longuet-Higgins Best Paper Award of the European Conference on Computer Vision (ECCV), the 2006 Olympus Prize of the German Pattern Recognition Society (DAGM), the 2006 Outstanding Dissertation Award of the German Society of Computer Science (GI), the Swiss Computer Science Society (SI), the Austrian Computer Society (OCG) and the German Chapters of the ACM (GChACM) as well as the 2006 Dr. Eduard Martin Prize of the Saarland University. Moreover, he was nominated as one of two national candidates for the 2008 Cor Baayen Award of the European Research Consortium for Informatics and Mathematics (ERCIM).

Course Material

PART I: Variational Basics
PART II: Modeling Aspects
PART III: Efficient Numerics

Andrés Bruhn / September 23, 2009