Image Processing of Tensor Fields

Matrix-valued data sets (so-called tensor fields) are gaining increasing importance in digital imaging. This has been triggered by the following developments:

• Diffusion tensor magnetic resonance imaging (DT-MRI) by now is an established tool for medical diagnosis. DT-MRI is a 3-D imaging method that yields a diffusion tensor in each voxel. This diffusion tensor describes the diffusive behaviour of water molecules in the tissue. This tensor can be represented by a positive semidefinite 3 x 3 matrix in each voxel.


• Tensors have shown their use as a general tool in image analysis, segmentation and grouping. This also includes widespread applications of the so-called structure tensor in fields ranging from motion analysis to texture segmentation.


• A number of applications in science and engineering produce tensor fields: The tensor concept is a common physical description of anisotropic behaviour, especially in solid mechanics and civil engineering (e.g. stress-strain relationships, inertia tensors, diffusion tensors, permittivity tensors).

Diffusion Filters for Tensor Fields

We have proposed various tensor-valued isotropic and anisotropic diffusion filter concepts:

• One class of concepts treats every tensor channel componentwise, in the spirit of Di Zenzo's approach to multichannel images, here however all channels share the structural information synthetised in a joint diffusion tensor (channel coupling). This construction indeed allows for smoothing preferently along discontinuities of the tensor field, while smoothing across discontinuities is inhibited. We have shown that suitable discretisations of these processes preserve the positive semidefiniteness of the initial tensor fields [1], [16].
• Another class of concepts is based on a novel operator-algebraic framework for matrix fields. We have proposed truly matrix-valued diffusion processes that automatically ensure appropriate channel interaction, therefore avoiding the need of explicit channel coupling. In order to solve these novel PDEs we proposed and elaborated matrix-valued numerical schemes [23], [24], [31], [33].

Filtering, Regularisation and Interpolation of Tensor Fields

• Novel energy functionals have been presented for the variational restoration of noisy tensor fields. Their Euler-Lagrange equations can be regarded as fully implicit time discretisations of tensor-valued anisotropic diffusion filters [1].
• Median filters belong to the most popular nonlinear filters for image denoising. Usual median filters require an ordering of the input data. This is not possible in the tensor-valued setting. We have thus expoited another property of the scalar-valued median: The median minimises the sum of distances to all input data. This interpretation can be generalised to the matrix-valued case by considering distances induced by the Frobenius norm. The resulting median filter is robust under noise and preserves discontinuities of the tensor field [6]. Extensions and algorithmic aspects are discussed in [11], [18], [19].
• We have proposed a unified framework for interpolation and regularisation of scalar- and tensor-valued images based on elliptic PDEs [17]. We showed that a rotationally invariant interpolation model based on anisotropic diffusion with a diffusion tensor as in [1], [16] outperforms interpolants with radial basis functions, it allows discontinuity-preserving interpolation, and it respects positive semidefiniteness of the input tensor data.
• In [28] we have proposed a discrete variational approach for the filtering of matrix fields. The model is based on nonlocal data similarity and smoothness contraints and it can be seen as the matrix-valued extension of the NDS filter for scalar images proposed by Mrázek, Weickert and Bruhn in 2006. The proposed filter contains novel counterparts of M-smoothing and bilateral filtering for matrix fields. In addition, other existing methods such as the affine-invariant (Pennec et al. 2006) and the log-Euclidean (Fillard et al. 2007) regularisation of tensor fields are special cases of the proposed filtering framework.

Operator-Algebraic Framework for Matrix Fields

• For the processing of matrix fields we proposed a general method to transfer calculus concepts for scalar images to the setting of matrix fields. The symmetric matrices are regarded as operators generalising real numbers with a relatively rich underlying algebraic stucture. This operator-algebraic framework allows for an efficient transfer of scalar image processing methodologies such as PDE-based and variational methods as well as their numerical solution schemes to the matrix-valued setting [23], [24], [26].
• This methodology has been succesfully employed to develop matrix-valued counterparts of nonlinear diffusion processes [23], [24], [31], regularisation approaches [25], [32], inverse scale-space techniques [33], and PDE-based mathematical morphology [35], [38].

Structure Tensors

• The structure tensor is a classical tool in image processing and computer vision (Förstner/Gülch 1987, Rao/Schunck 1990, Bigün et al. 1991). It averages orientations by means of Gaussian smoothing of all tensor channels. This may be regarded as tensor-valued linear diffusion filtering. In order to allow preservation of discontinuities, we have replaced this linear diffusion process by a nonlinear tensor-valued diffusion [1], [10], creating thus the so-called nonlinear structure tensor. Applications in the fields of optical flow estimation [2], texture analysis [3], and segmentation [4,5], demonstrate the favourable performance of the resulting nonlinear structure tensor. A comparison and evaluation of the nonlinear structure tensor in the context of other adaptive structure tensor concepts has been presented in [14].
• In [1], [7] we have generalised the concept of image gradient to the tensor-valued setting by extending Di Zenso's method for vector-valued data. Each channel considered as independent scalar image gives rise to a structure tensor, then these structures tensors are summed up to build a novel structure tensor for multi channel images that we refer to as Weickert's structure tensor.
  
• This construction has been refined to a customisable structure tensor in [21]. There the resulting structure tensor is a weighted sum of tensors of scalar quantities that are now not just the channels, but other meaningful scalar quantities derived from the matrix field. The weights are provided by the user, and depending on the choice of weights the emerging structure tensor has a sensitivity for certain features of the matrix field. A special constellation of the weights turns the customisable structure tensor into Weickert's structure tensor.
• Based on the operator-algebraic framework we have developed a novel structure tensor concept for matrix fields, the so-called full structure tensor [31]. With this tensor we are able to deduce directional information in matrix fields with the additional possibility to introduce prior structural knowledge. By a natural reduction operation Weickert's structure tensor appears as a special case of the full structure tensor.
• It is a known limitation of the traditional structure tensor (Förstner/Gülch 1987) that it can only represent a single dominant orientation. We have developed its generalisation to a higher-order tensor model. The proposed higher-order structure tensor is able to capture the orientations of more complex neighborhoods, for example corners, junctions, and multivalued images. The tensor order allows to specify the maximum complexity the structure tensor can represent and can be chosen based on the requirements of a given application [30].

Tensor-Valued Morphology

• Algebraic-geometric approaches. Dilation/erosion rely on maximum and minimum operations. Their generalisation to the tensor-valued setting is not straightforward due to the lack of suitable ordering relations for symmetric matrices. In [8] we have proposed such extensions exploiting analytic-algebraic and geometric properties of symmetric matrices while preserving positive definiteness of the initial data. Unfortunately the proposed approaches have their weaknesses when it comes to computational issues. This led to the development of more appropriate extensions of dilation and erosion operations for matrix-valued data.
• Ordering-based approaches. In the matrix valued setting there is a partial ordering, the so-called Loewner ordering, that is suitable for the definition of appropriate maximal and minimal elements of a set of matrices. The task to determine these extremal elements is already non-trivial in the case of 2 by 2 matrices [15] and becomes more intricate in higher dimensions [12], [20]; it requires tools from convex analysis and computational geometry. Nevertheless, the dilation and erosion operations inferred from the ordering approach exhibit the desired rotational invariance, preservation of positive definiteness, and continuous dependence on the initial matrix data. This makes it possible to define morphological derivatives and shock filters for matrix fields.
• PDE-based approaches. Dilation/erosion can be described by nonlinear hyperbolic first order PDEs as introduced by R. van den Boomgaard in 1992. Based on novel mathematical notions for symmetric matrices, in [22] we were able to find the truly matrix-valued counterpart to these nonlinear morphological PDEs. For the appropriate numerical treatment of those novel matrix-valued PDEs we devised matrix-valued generalisations of known real-valued schemes, see especially [35]. More recently, we have developed adaptive, anisotropic PDEs for matrix-valued morphology. The anisotropic evolution is guided by a steering tensor whose construction relies on the full structure tensor for matrix data. In order to enable proper directional steering we proposed directional matrix-valued numerical schemes based on directional finite differences via interpolation. We demonstrated that these PDEs are capable, for instance, to fill in missing data and to complete directional structures [36], [37], [38].

Tensor-Valued Level Set Methods

• Based on Weickert's structure tensor we have proposed tensor-valued extensions of mean curvature motion, self-snakes and geodesic active contours. Our experiments demonstrated that the tensor-valued level set methods inherit essential properties from their scalar-valued counterparts and that they are highly robust under noise, both in the two-dimensional [7] and three-dimensional setting [9]. See also the survey [16].
• Based on the full structure tensor for matrix fields we have proposed truly matrix-valued PDEs for mean curvature motion and self-snakes. Experiments on synthetic and positive semidefinite DT-MRI data illustrated that the matrix-valued methods inherit desirable properties of their scalar-valued predecessors, e.g. very good denoising capabilities combined with feature preserving qualities [23].

Registration of Tensor Fields

• A variational framework for the registration of tensor-valued images has been proposed in [27], [34]. It is based on an energy functional with four terms: a data term based on a diffusion tensor constancy constraint, a compatibility term encoding the physical model linking domain deformations and tensor reorientation, and smoothness terms for deformation and tensor orientation. This approach has been specially designed for the registration of DT-MRI data. However, it can be easily adapted to different tensor deformation models for registering other types of tensor-valued data.

Organisation of Workshops

• 1st Workshop on Visualisation and Image Processing of Tensor Fields,
  Dagstuhl, Germany, April 18-22, 2004. The workshop led to a book [13] which is the first edited book covering visualisation and processing of tensor fields.
• 2nd Workshop on Visualization and Processing of Tensor Fields.
  Dagstuhl, Germany, January 9-13, 2007. The workshop led to a book [29].
• 3rd Workshop on New Developments in the Visualization and Processing of Tensor Fields.
  Dagstuhl, Germany, July 19-24, 2009.

1. J. Weickert, T. Brox:
   Diffusion and regularization of vector- and matrix-valued images.
   In M. Z. Nashed, O. Scherzer (Eds.): Inverse Problems, Image Analysis, and Medical Imaging. Contemporary Mathematics, Vol. 313, 251-268, AMS, Providence, 2002.

2. T. Brox, J. Weickert:
   Nonlinear matrix diffusion for optic flow estimation.
   In L. Van Gool (Ed.): Pattern Recognition. Lecture Notes in Computer Science, Vol. 2449, Springer, Berlin, 446-453, 2002.

3. M. Rousson, T. Brox, R. Deriche:
   Active unsupervised texture segmentation on a diffusion based feature space.
   Technical report no. 4695, Odyssée, INRIA Sophia-Antipolis, France, 2003.
   Slightly extended version of the conference paper with the same title,
   Proc. 2003 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Madison, WI, 2003.

4. T. Brox, M. Rousson, R. Deriche, J. Weickert:
   Unsupervised segmentation incorporating colour, texture, and motion.
   In N. Petkov, M. A. Westenberg (Eds.): Computer Analysis of Images and Patterns. Lecture Notes in Computer Science, Vol. 2756, Springer, Berlin, 353-360, 2003.

5. T. Brox, M. Rousson, R. Deriche, J. Weickert:
   Unsupervised segmentation incorporating colour, texture, and motion.
   Technical report no. 4760, Odyssée, INRIA Sophia-Antipolis, France, 2003.

6. M. Welk, C. Feddern, B. Burgeth, J. Weickert:
   Median filtering of tensor-valued images.
   In B. Michaelis, G. Krell (Eds.): Pattern Recognition. Lecture Notes in Computer Science, Vol. 2781, Springer, Berlin, 17–24, 2003.

7. C. Feddern, J. Weickert, B. Burgeth:
   Level-set methods for tensor-valued images.
   In O. Faugeras, N. Paragios (Eds.): Proc. Second IEEE Workshop on Variational, Geometric and Level Set Methods in Computer Vision. Nice, France, pp. 65-72. INRIA, Oct. 2003.

8. B. Burgeth, M. Welk, C. Feddern, J. Weickert:
   Morphological operations on matrix-valued images.
   In T. Pajdla, J. Matas (Eds.): Computer Vision - ECCV 2004. Lecture Notes in Computer Science, Vol. 3024, 155-167, Springer, Berlin, 2004.

9. C. Feddern, J. Weickert, B. Burgeth, M. Welk:
   Curvature-driven PDE methods for matrix-valued images.
   International Journal of Computer Vision, Vol. 69, No. 1, 91-103, Aug. 2006.
   Revised version of Technical Report No. 104, Department of Mathematics, Saarland University, Saarbrücken, Germany, April 2004.
   

10. T. Brox, J. Weickert, B. Burgeth, P. Mrázek:
    Nonlinear structure tensors.
    Image and Vision Computing, Vol. 24, No. 1, 41-55, Jan. 2006.
    Revised version of Technical Report No. 113, Department of Mathematics, Saarland University, Saarbrücken, Germany, 2004.
    

11. M. Welk, F. Becker, C. Schnörr, J. Weickert:
    Matrix-valued filters as convex programs.
    In R. Kimmel, N. Sochen, J. Weickert (Eds.): Scale-Space and PDE Methods in Computer Vision. Lecture Notes in Computer Science, Vol. 3459, Springer, Berlin, 204–216, 2005.
    

12. B. Burgeth, N. Papenberg, A. Bruhn, M. Welk, C. Feddern, J. Weickert:
    Morphology for higher-dimensional tensor data via Loewner ordering.
    In C. Ronse, L. Najman, E. Decencière (Eds.): Mathematical Morphology: 40 Years On. Computational Imaging and Vision, Vol. 30, Springer, Dordrecht, 407–416, 2005.
    

13. J. Weickert, H. Hagen (Eds.):
    Visualization and Processing of Tensor Fields.
    Springer, Berlin, 2006.
    

14. T. Brox, R. van den Boomgaard, F. Lauze, J. van de Weijer, J. Weickert, P. Mrázek, P. Kornprobst:
    Adaptive structure tensors and their applications.
    In J. Weickert, H. Hagen (Eds.): Visualization and Processing of Tensor Fields, 17-47, Springer, Berlin, 2006.
    Revised version of Technical Report No. 141, Department of Mathematics, Saarland University, Saarbrücken, Germany, 2005.
    

15. B. Burgeth, M. Welk, C. Feddern, J. Weickert:
    Mathematical morphology on tensor data using the Loewner ordering.
    In J. Weickert, H. Hagen (Eds.): Visualization and Processing of Tensor Fields, 357-367, Springer, Berlin, 2006.
    Revised version of Technical Report No. 160, Department of Mathematics, Saarland University, Saarbrücken, Germany, 2005.
    

16. J. Weickert, C. Feddern, M. Welk, B. Burgeth, T. Brox:
    PDEs for tensor image processing.
    In J. Weickert, H. Hagen (Eds.): Visualization and Processing of Tensor Fields, 399-414, Springer, Berlin, 2006.
    Revised version of Technical Report No. 143, Department of Mathematics, Saarland University, Saarbrücken, Germany, 2005.
    

17. J. Weickert, M. Welk:
    Tensor field interpolation with PDEs.
    In J. Weickert, H. Hagen (Eds.): Visualization and Processing of Tensor Fields, 315-325, Springer, Berlin, 2006.
    Revised version of Technical Report No. 142, Department of Mathematics, Saarland University, Saarbrücken, Germany, 2005.
    

18. M. Welk, C. Feddern, B. Burgeth, J. Weickert:
    Tensor median filtering and M-smoothing.
    In J. Weickert, H. Hagen (Eds.): Visualization and Processing of Tensor Fields, 345-356, Springer, Berlin, 2006.
    

19. M. Welk, J. Weickert, F. Becker, C. Schnörr, C. Feddern, B. Burgeth:
    Median and related local filters for tensor-valued images.
    Signal Processing, Vol. 87, No. 2, 291-308, February 2007.
    Revised version of Technical Report No. 135, Department of Mathematics, Saarland University, Saarbrücken, Germany, April 2005.
    

20. B. Burgeth, N. Papenberg, A. Bruhn, M. Welk, J. Weickert:
    Mathematical morphology for matrix fields induced by the Loewner ordering in higher dimensions.
    Signal Processing, Vol. 87, No. 2, 277-290, February 2007.
    Revised version of Technical Report No. 161, Department of Mathematics, Saarland University, Saarbrücken, Germany, December 2005.
    

21. T. Schultz, B. Burgeth, J. Weickert:
    Flexible segmentation and smoothing of DT-MRI fields through a customizable structure tensor.
    In G. Bebis, R. Boyle, B. Parvin, D. Koracin, P. Remagnino, A. V. Nefian, M. Gopi, V. Pascucci, J. Zara, J. Molineros, H. Theisel, T. Malzbender (Eds.): Advances in Visual Computing. Lecture Notes in Computer Science, Vol. 4291, Springer, 454-464, 2006.
    Awarded the ISVC 2006 Best Paper Award.
    

22. B. Burgeth, A. Bruhn, S. Didas, J. Weickert, M. Welk:
    Morphology for tensor data: Ordering versus PDE-based approach.
    Image and Vision Computing, Vol. 25, No. 4, 496-511, 2007.
    Revised version of Technical Report No. 162, Department of Mathematics, Saarland University, Saarbrücken, Germany, December 2005.
    

23. B. Burgeth, S. Didas, L. Florack, J. Weickert:
    A generic approach to the filtering of matrix fields with singular PDEs.
    In F. Sgallari, A. Murli, N. Paragios (Eds.), Scale Space and Variational Methods in Computer Vision. Lecture Notes in Computer Science, Vol. 4485, 556-567, Springer, Berlin, 2007.

24. B. Burgeth, S. Didas, L. Florack, J. Weickert:
    A generic approach to diffusion filtering of matrix-fields.
    Computing, Vol. 81, No. 2-3, 179-197, Nov. 2007. Invited Paper.
    

25. O. Christiansen, J. Lie, B. Burgeth:
    A dual operator algebraic method for regularization of matrix valued images.
    Accepted for publication in Inverse Problems and Imaging, 2007.
    

26. B. Burgeth:
    Partial Differential Equations for Scale Space Analysis and Matrix Field Processing.
    Habilitation Thesis. Faculty of Mathematics and Computer Science, August 2008. Available upon request.
    

27. S. Barbieri, M. Welk, J. Weickert:
    Variational registration of tensor-valued images.
    Proc. CVPR Workshop »Tensors in Image Processing and Computer Vision«, Anchorage, Alaska, USA, 23 June 2008, pages 1-6.
    

28. L. Pizarro, B. Burgeth, S. Didas, J. Weickert:
    A generic neighbourhood filtering framework for matrix fields.
    In D. Forsyth, P. Torr, A. Zisserman (Eds.): Computer Vision – ECCV 2008. Lecture Notes in Computer Science, Vol. 5304, 521-532. Springer, Berlin, 2008.
    

29. D. H. Laidlaw, J. Weickert (Eds.):
    Visualization and Processing of Tensor Fields: Advances and Perspectives
    Springer, Berlin, 2009.

30. T. Schultz, J. Weickert, H.-P. Seidel:
    A higher-order structure tensor.
    In D. H. Laidlaw, J. Weickert (Eds.): Visualization and Processing of Tensor Fields: Advances and Perspectives. Springer, Berlin, 263-279, 2009.
    Revised version of Research Report MPI-I-2007-4-005, Max-Planck-Institut für Informatik, Saarbrücken, Germany, July 2007.

31. B. Burgeth, S. Didas, J. Weickert:
    A general structure tensor concept and coherence-enhancing diffusion filtering for matrix fields.
    In D. H. Laidlaw, J. Weickert (Eds.): Visualization and Processing of Tensor Fields: Advances and Perspectives. Springer, Berlin, 305-323, 2009.
    Revised version of Technical Report No. 197, Department of Mathematics, Saarland University, Saarbrücken, Germany, July 2007.

32. S. Setzer, G. Steidl, B. Popilka, B. Burgeth:
    Variational methods for denoising matrix fields.
    In D. H. Laidlaw, J. Weickert (Eds.): Visualization and Processing of Tensor Fields: Advances and Perspectives. Springer, Berlin, 341-360, 2009.

33. J. Lie, B. Burgeth, O. Christiansen:
    An operator algebraic inverse scale space method for symmetric matrix valued images.
    In D. H. Laidlaw, J. Weickert (Eds.): Visualization and Processing of Tensor Fields: Advances and Perspectives. Springer, Berlin, 361-376, 2009.

34. S. Barbieri, M. Welk, J. Weickert:
    A variational approach to the registration of tensor-valued images.
    In S. Aja-Fernandez, R. de Luis-Garcia, D. Tao, X. Li (Eds.): Tensors in Image Processing and Computer Vision, pages 59-77, Springer, London, 2009.
    Revised version of Technical Report No. 221, Department of Mathematics, Saarland University, Saarbrücken, Germany, September 2008.

35. B. Burgeth, M. Breuß, S. Didas, J. Weickert:
    PDE-based morphology for matrix fields: Numerical solution schemes.
    In S. Aja-Fernandez, R. de Luis-Garcia, D. Tao, X. Li (Eds.): Tensors in Image Processing and Computer Vision, pages 125-150, Springer, London, 2009.
    Revised version of Technical Report No. 220, Department of Mathematics, Saarland University, Saarbrücken, Germany, September 2008.

36. B. Burgeth, M. Breuß, L. Pizarro, J. Weickert:
    PDE-driven adaptive morphology for matrix fields.
    In X.-C. Tai, K. Mørken, M. Lysaker, K.-A. Lie (Eds.): Scale Space and Variational Methods in Computer Vision. Lecture Notes in Computer Science, Vol. 5567, 247-258, Springer, Berlin, 2009.

37. L. Pizarro, B. Burgeth, M. Breuß, J. Weickert:
    A directional Rouy-Tourin scheme for adaptive matrix-valued morphology.
    In M.H.F. Wilkinson and J.B.T.M. Roerdink (Eds.): Proceedings of Ninth International Symposium on Mathematical Morphology, Lecture Notes in Computer Science, Vol. 5720, 250-260, Springer, Berlin, 2009.

38. B. Burgeth, L. Pizarro, M. Breuß, J. Weickert:
    Adaptive continuous-scale morphology for matrix fields.
    To appear in International Journal of Computer Vision.
    Revised version of Technical Report No. 237, Department of Mathematics, Saarland University, Saarbrücken, Germany, May 2009.

MIA Group
©2001-2023
The author is not
responsible for
the content of
external pages.

Imprint - Data protection