Image Segmentation

The partitioning of an image into meaningful regions is one of the principal computer vision tasks. On the way from the raw pixel data up to a meaning of the image, it is necessary to reduce the amount of information and to represent it in a compact manner. In this sense, image segmentation plays a key role, as it opens the possibility to work with a few regions instead of huge amounts of pixels.

However, just because image segmentation is so important for image understanding and linked with almost all the other tasks in computer vision, it is also very difficult to define what it actually should do. What is a meaningful region?

It depends. We want to find regions that we expect to find. We want the regions to correspond to objects that we know, that we have seen before. On the other hand, the regions we want to find should still be in compliance with the information given my the raw pixels. The inside of regions should be as similar as possible and the content of two different regions should be as dissimilar as possible.

So we realize that image segmentation is the chain link between data-driven low-level vision and knowledge-driven high-level vision. Good low-level vision techniques are necessary to make the best out of the information we get from the image. Having a good representation and a large amount of knowledge is necessary to find what we want to find. Putting both together is a difficult task and research is far from being successful in the general case.

Our research in this field benfits from the experience of our group on image processing and low-level vision. Many techniques from our other research areas can be used for the segmentation task. Therefore, our main focus is on unsupervised, data-driven segmentation. The research on this topic is in cooperation with the Odyssée group at INRIA Sophia-Antipolis in France.

• Texture Segmentation
  One challenging task for data-driven segmentation is the handling of texture. We proposed a novel texture feature space based on the second moment matrix smoothed by nonlinear diffusion. We call the resulting matrix nonlinear structure tensor pursuant to the classic structure tensor based on Gaussian smoothing. Good results could be achieved with a two-region segmentation on textured images [1].
  The feature space can be further improved by including a measure for the texture's scale. We developed a region based scale measure which exploits the special properties of total variational flow. Combined with the texture features obtained from the nonlinear structure tensor improved segmentation results can be obtained [4,6].
  

• Multi-Cue Integration
  To succeed in data-driven segmentation it is vital to use all available information provided by the image. Our variational approach allows to supplement additional features besides the gray value, such as color, texture, and motion. Smoothing the features by coupled nonlinear diffusion allows to improve the coherence between the channels. It also implies an implicit weighting of the features by their tendency to create homogeneous regions. Also the nonparametric statistical model we use to describe regions implies an implicit weighting by the ability of a feature channel to separate regions. This might be an answer to the important question, how to choose appropriate weights for the fusion of several features [2,3].

• Multi-Scale Approach
  Image segmentation is a problem that has many local solutions. In order to find the global optimum (according to a given model) it is a well-known procedure to first consider coarser versions of the problem and to solve the fine problem step by step with a coarse-to-fine strategy. We applied this idea to level set based segmentation and found out, that the dependency of the result on the initialization of the level set function decreases considerably [3].

• Multiple Object Tracking
  It is possible to apply the same segmentation algorithm to tracking. Indeed segmentation and tracking are closely related topics. Assuming that there are no temporal occlusions of the tracked objects, tracking is even easier to perform than segmentation, because the number of objects as well as their initial position and shape are known at the beginning. In order to be able to track more than one object without overlapping we developed a new scheme with coupled level set functions [2,3].
  

• Multiple-Region Segmentation
  In order to extend also the segmentation approach to multiple regions one has to find a solution for the problem how to determine the number of objects in the image as well their coarse position and shape. For this purpose we introduced a hierarchical approach which uses the two-region segmentation framework in order to successively split regions until further splits do not decrease the energy anymore. With this strategy one obtains both an estimate for the number of regions as well as reasonable initialisations. Afterwards the coupled scheme already used for multiple object tracking is employed in order to improve the exact positioning of the contours [5].

1. M. Rousson, T. Brox, R. Deriche:
   Active unsupervised texture segmentation on a diffusion based feature space.
   Technical report no. 4695, Odyssée, INRIA Sophia-Antipolis, France, 2003.
   Slighly extended version of the conference paper with the same title,
   Proc. 2003 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Vol. 2, pp. 699-704, Madison, WI, 2003.
   

2. T. Brox, M. Rousson, R. Deriche, J. Weickert:
   Unsupervised segmentation incorporating colour, texture, and motion.
   In N. Petkov, M. A. Westenberg (Eds.): Computer Analysis of Images and Patterns. Lecture Notes in Computer Science, Vol. 2756, Springer, Berlin, 353-360, 2003.
   

3. T. Brox, M. Rousson, R. Deriche, J. Weickert:
   Unsupervised segmentation incorporating colour, texture, and motion.
   Technical report no. 4760, Odyssée, INRIA Sophia-Antipolis, France, 2003.
   

4. T. Brox, J. Weickert:
   A TV Flow Based Local Scale Measure for Texture Discrimination .
   In T. Pajdla, J. Matas (Eds.): Computer Vision - ECCV 2004. Lecture Notes in Computer Science, Vol.3022, Springer, Berlin, 578-590, 2004.
   

5. T. Brox, J. Weickert:
   Level Set Based Image Segmentation with Multiple Regions .
   In C. E. Rasmussen, H. H. Bülthoff, M. A. Giese, B. Schölkopf (Eds.): Pattern Recognition. Lecture Notes in Computer Science, Vol. 3175, Springer, Berlin, 415-423, 2004.
   

6. T. Brox, J. Weickert:
   A TV Flow Based Scale Estimate and its Application to Texture Discrimination
   Technical Report No. 134, Department of Mathematics, Saarland University, Saarbrücken, Germany, 2005.
   

MIA Group
©2001-2023
The author is not
responsible for
the content of
external pages.

Imprint - Data protection