Optic Flow

Correspondence problems are one of the key problems in computer vision. They appear for example in motion analysis, image matching, or stereo vision. The goal is to assign structures in one image to structures in a second image, in a way that corresponding structures are as similar as possible. In the case of motion analysis, for instance, this means that pixels are tracked in a sequence by finding the shift vector for each pixel from one image to the next. For this, some property of the pixels has to be assumed to stay constant over time.
Correspondence problems are in general ill-posed, as everybody can imagine there are several possibilities to match structures. To deal with this problem additional smoothness assumptions have to be applied, which can considerably affect the outcome.

This figure shows two frames of the famous Yosemite test sequence and the shift vectors computed by one of our algorithms. Here the shift vectors are shown in a coloured orientation plot where the colour represents the orientation of the vector and brightness stands for its magnitude.

In our group, we focus mainly on motion analysis, in particular on variational approaches and their numerical realisation. But we also look into the very related areas of image matching and stereo vision.

Smoothness Assumptions for Variational Optic Flow Estimation

During the years, a large amount of different smoothness assumptions have been proposed in the literature. This goes from homogeneous, over image-driven to flow-driven smoothness terms, which can again be both isotropic or anisotropic. A classification of all these techniques helps to understand similarities and differences of methods much better [1].

Combination of Local and Global Methods

There are two different ways to motion estimation. In local methods the optic flow within a neighbourhood is assumed to be smooth and the flow is computed also respecting only this neighbourhood. In global methods, on the other hand, a global energy functional is minimised that respects all the given data to determine the resulting flow field. Both approaches have their advantages and drawbacks. We succeeded in combining both strategies, what allows to take over the best properties from both worlds. So we are able to ensure dense flow fields which are still robust under noise [2,3].

Optic Flow Estimation with the Nonlinear Structure Tensor

Local methods are often based on the so-called structure tensor. This structure tensor is obtained by smoothing the outer product of the image gradient with a Gaussian kernel. Instead we apply nonlinear diffusion for the smoothing, therefore using the nonlinear structure tensor. This leads to a higher accuracy in the estimation [4].

Higher Order Optic Flow Constraints

Real world image sequences often suffer under slight brightness changes. Applying the standard optic flow constraint, which assumes the grey value of corresponding pixels to stay constant from one image to the next, is not a good way to go. This is because due to the bringhtness change, the basic assumption of this constraint is not valid anymore. Therefore, we apply measures that are less dependent on brightness, such as the image gradient or the Laplacian [5].

Theory for Warping

Warping has become a very popular strategy for optic flow estimation in the presence of large displacements. So far warping has only algorithmically been motivated. We found out, that warping can also be seen as a numerical scheme which solve the original non-linearised constancy assumption, instead of the widely used linearised term. The consequent implementation of this scheme leads to very accurate results like those shown in the images above [6].

Fast Numerical Algorithms

Variational approaches yield large linear systems, which results in long computation times, if implemented in a naive way. In our group we developed many techniques that speed up the computation considerably. See also our research page on scientific computing.

1. J. Weickert, C. Schnörr:
   A theoretical framework for convex regularizers in PDE-based computation of image motion.
   International Journal of Computer Vision, Vol. 45, No. 3, 245-264, December 2001.
   Revised version of Technical Report No. 13/2000, Computer Science Series, University of Mannheim, Germany, June 2000.

2. A. Bruhn, J. Weickert, C. Schnörr:
   Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods.
   International Journal of Computer Vision, in press.
   Revised version of Technical Report No. 82, Department of Mathematics, Saarland University, Saarbrücken, Germany, April 2003.
   

3. A. Bruhn, J. Weickert, C. Schnörr:
   Combining the advantages of local and global optic flow methods.
   In L. Van Gool (Ed.): Pattern Recognition. Lecture Notes in Computer Science, Vol. 2449, Springer, Berlin, 454-462, 2002.

4. T. Brox, J. Weickert:
   Nonlinear matrix diffusion for optic flow estimation,
   In L. Van Gool (Ed.): Pattern Recognition. Lecture Notes in Computer Science, Vol. 2449, Springer, Berlin, 446-453, 2002.

5. J. Weickert, A. Bruhn, N. Papenberg, T. Brox:
   Variational Optic Flow Computation: From Continuous Models to Algorithms.
   In L. Alvarez (Ed.): IWCVIA '03: International Workshop on Computer Vision and Image Analysis, Universidad de Las Palmas de Gran Canaria, Dec. 4-6, 2003.
   

6. T. Brox, A. Bruhn, N. Papenberg, J. Weickert:
   High accuracy optical flow estimation based on a theory for warping.
   In T. Pajdla, J. Matas (Eds.): Computer Vision - ECCV 2004. Lecture Notes in Computer Science, Vol.3024, Springer, Berlin, 25-36, 2004.

MIA Group
©2001-2005
The author is not
responsible for
the content of
external pages.