The partitioning of an image into meaningful regions is one of the
principal computer vision tasks. On the way from the raw pixel data
up to a meaning of the image, it is necessary to reduce the amount
of information and to represent it in a compact manner. In this sense,
image segmentation plays a key role, as it opens the possibility to work
with a few regions instead of huge amounts of pixels.
However, just because image segmentation is so important for image
understanding and linked with almost all the other tasks in computer
vision, it is also very difficult to define what it actually should do.
What is a meaningful region?
It depends. We want to find regions that we expect to find. We want the
regions to correspond to objects that we know, that we have seen before.
On the other hand, the regions we want to find should still be in
compliance with the information given my the raw pixels. The inside of
regions should be as similar as possible and the content of two different
regions should be as dissimilar as possible.
So we realize that image segmentation is the chain link between
data-driven low-level vision and knowledge-driven high-level vision.
Good low-level vision techniques are necessary to make the best out of the
information we get from the image. Having a good representation
and a large amount of knowledge is necessary to find what we want to find.
Putting both together is a difficult task and research is far from being
successful in the general case.
Our research in this field benfits from the experience of our group
on image processing and low-level vision. Many techniques from our
other research areas can be used for the segmentation task. Therefore,
our main focus is on unsupervised, data-driven segmentation. The research
on this topic is in cooperation with the Odyssée group at
INRIA Sophia-Antipolis in France.
-
Texture Segmentation
One challenging task for data-driven segmentation is the handling
of texture. We proposed a novel texture feature space based on
the second moment matrix smoothed by nonlinear diffusion. We call
the resulting matrix nonlinear structure tensor pursuant to
the classic structure tensor based on Gaussian smoothing. Good
results could be achieved with a two-region segmentation on textured
images [1].
The feature space can be further improved by including a measure for
the texture's scale. We developed a region based scale measure which
exploits the special properties of total variational flow. Combined
with the texture features obtained from the nonlinear structure tensor
improved segmentation results can be obtained [4,6].
-
Multi-Cue Integration
To succeed in data-driven segmentation it is vital to use all
available information provided by the image. Our variational
approach allows to supplement additional features besides the
gray value, such as color, texture, and motion. Smoothing the
features by coupled nonlinear diffusion allows to improve the
coherence between the channels. It also implies an implicit weighting
of the features by their tendency to create homogeneous regions.
Also the nonparametric statistical model we use to describe regions
implies an implicit weighting by the ability of a feature channel
to separate regions. This might be an answer to the important
question, how to choose appropriate weights for the fusion of
several features [2,3].
-
Multi-Scale Approach
Image segmentation is a problem that has many local solutions.
In order to find the global optimum (according to a given model)
it is a well-known procedure to first consider coarser versions of
the problem and to solve the fine problem step by step with a
coarse-to-fine strategy. We applied this idea to level set based
segmentation and found out, that the dependency of the result on
the initialization of the level set function decreases considerably
[3].
-
Multiple Object Tracking
It is possible to apply the same segmentation algorithm to tracking.
Indeed segmentation and tracking are closely related topics. Assuming
that there are no temporal occlusions of the tracked objects, tracking
is even easier to perform than segmentation, because the number of objects
as well as their initial position and shape are known at the beginning.
In order to be able to track more than one object without overlapping
we developed a new scheme with coupled level set functions
[2,3].
-
Multiple-Region Segmentation
In order to extend also the segmentation approach to multiple regions
one has to find a solution for the problem how to determine the number
of objects in the image as well their coarse position and shape. For
this purpose we introduced a hierarchical approach which uses the
two-region segmentation framework in order to successively split regions
until further splits do not decrease the energy anymore. With this
strategy one obtains both an estimate for the number of regions as well
as reasonable initialisations. Afterwards the coupled scheme already used
for multiple object tracking is employed in order to improve the exact
positioning of the contours
[5].
-
M. Rousson, T. Brox, R. Deriche:
Active unsupervised texture segmentation on a diffusion based
feature space.
Technical report no. 4695, Odyssée, INRIA Sophia-Antipolis,
France, 2003.
Slighly extended version of the conference paper with the same
title,
Proc. 2003 IEEE Computer Society Conf. on Computer Vision and Pattern
Recognition, Vol. 2, pp. 699-704, Madison, WI, 2003.
-
T. Brox, M. Rousson, R. Deriche, J. Weickert:
Unsupervised segmentation incorporating colour, texture, and
motion.
In N. Petkov, M. A. Westenberg (Eds.): Computer Analysis of Images
and Patterns. Lecture Notes in Computer Science, Vol. 2756, Springer,
Berlin, 353-360, 2003.
-
T. Brox, M. Rousson, R. Deriche, J. Weickert:
Unsupervised segmentation incorporating colour, texture, and
motion.
Technical report no. 4760, Odyssée, INRIA Sophia-Antipolis,
France, 2003.
-
T. Brox, J. Weickert:
A TV Flow Based Local Scale Measure for Texture Discrimination
.
In T. Pajdla, J. Matas (Eds.): Computer Vision - ECCV 2004.
Lecture Notes in Computer Science, Vol.3022, Springer, Berlin, 578-590, 2004.
-
T. Brox, J. Weickert:
Level Set Based Image Segmentation with Multiple Regions
.
In C. E. Rasmussen, H. H. Bülthoff, M. A. Giese, B. Schölkopf (Eds.): Pattern
Recognition. Lecture Notes in Computer Science, Vol. 3175, Springer, Berlin,
415-423, 2004.
-
T. Brox, J. Weickert:
A TV Flow Based Scale Estimate and its Application to Texture Discrimination
Technical Report No. 134, Department of Mathematics,
Saarland University, Saarbrücken, Germany, 2005.
|