Correspondence Problems in Computer Vision

Computer Science Teaching Award (summer term 2008)

Lecturer: Sebastian Hoffmann
Office hours: by appointment.

Examiner: Prof. Dr. Joachim Weickert

Summer Term 2015

Lectures (2h) with programming/theoretical exercises (2h)
(6 credit points)


Lectures: Mondays 16-18 c.t.
Lectures: building E 1.3, lecture hall 001

First lecture: Friday, April 24, 2015, 12-14 c.t.

Tutorials: 2 hours each week, Friday, 12-14 c.t.
Tutorials: either building E1.3, lecture hall 001 (theory)
Tutorials: or building E1.3, CIP-pool 104 (programming)

First tutorial: Friday, May 08, 2015 (programming)

29/09/2015:	The results of the second written exam are available here.
10/08/2015:	The results of the first written exam are available here.
21/07/2015:	A sample exam can be found here.
04/05/2015:	The registration is closed.
24/04/2015:	The registration is open, see here.
10/02/2015:	The first lecture is shifted to Friday, April 24, 12-14 c.t., building E 1.3, lecture hall 001, to avoid colisions with introductory meetings.
10/02/2015:	The course homepage is online. Important news will be posted here.

---

Synopsis – Prerequisites – Registration – Contents – Tutorials and Assignments – Exams – References

---

Example - Stereo Reconstruction

Example - Motion Estimation

Correspondence problems are a central topic in computer vision. The basic task amounts to identifying and matching corresponding features in different images/views of the same scene. Typical examples for correspondence problems are (i) the estimation of motion information from consecutive frames of an image sequence (optic flow), (ii) the reconstruction of a 3-D scene from a stereo image pair and (iii) the registration of medical image data from different image acquisition devices (e.g. CT and MRT).
The central part of this lecture is concerned with discussing the most important correspondence problems together with suitable algorithms for their solution.

Suited for students of visual computing, applied mathematics and computer science. Requires undergraduate knowledge in mathematics (e.g. ''Mathematik für Informatiker I-III'') and elementary C knowledge (for the programming assignments). Knowledge in image processing or differential equations is useful. The lectures will be given in English.

In order to participate in the lecture/exam you have to register twice:

• Firstly, you have to register online to the system of the MIA group. This gives us an estimate of the number of students attending the course and allows us to issue the certificates at the end of the semester. You could have registered for this lecture between April, 24, 2015, and May 4, 2015.
• Secondly, depending on your field of study, you have to register online for the lecture/exam in the general system of Saarland University: the HISPOS system.

Participants of the course can download the lecture materials here shortly before the lecture (access is password-protected). However, be aware that these slides are only provided to support the classroom teaching, not to replace it. Additional organisational information, such as examples and explanations that may be helpful or necessary to understand the content of the course (and thus relevant for the exam), will be provided in the lectures. It is solely your responsibility - not ours - to make sure that you receive this infomation. The following table shows a preliminary list of topics that will be covered during the semester.

Date	Lecture	Topic
24/04	Lecture 01	Introduction, Overview
27/04	Lecture 02	Block Matching, Correlation Methods, Occlusion Detection, Interest Points, Feature Methods
04/05	Lecture 03	Optic Flow I: Local Differential Methods, Parametrisation Models
11/05	Lecture 04	Optic Flow II: Global Differential Methods, Horn and Schunck
18/05	Lecture 05	Optic Flow III: Advanced Constancy Assumptions, Large Motion
01/06	Lecture 06	Optic Flow IV: Advanced Data and Smoothness Terms
08/06	Lecture 07	Optic Flow V: High Accuracy Methods
15/06	Lecture 08	Optic Flow VI: Advanced Numerics
22/06	Lecture 09	Stereo Matching I: Projective and Epipolar Geometry
29/06	Lecture 10	Stereo Matching II: Estimation of the Fundamental Matrix
06/07	Lecture 11	Optic Flow + Stereo: Scene Flow Estimation
13/07	Lecture 12	Medical Image Registration: Mutual Information, Elastic and Curvature Based Registration, Landmarks
20/07	Lecture 13	Particle Image Velocimetry: Div-Curl-Regularisation, Incompressible Navier Stokes Prior
27/07	Lecture 14	Summary, Outlook

The weekly tutorials alternate between theory and programming. They take place in:

Building E1.3, lecture hall 001 (theory)
Building E1.3, CIP-pool 104 (programming)

Programming excercises and theoretical assignments are offered as part of the tutorials. The regular attendance of these tutorials is required for the admission to the exams. You will not be allowed to take part in the exams if you miss more than two tutorials.

The assignments,the source code needed for the programming assignments as well as example solutions for all the assignments will be provided here during the semester.

Date	Assignment	Topic	Material	Solution
08/05	Assignment P01	Block Matching	sources	solution
15/05	Assignment T02	Sub-Pixel Refinement, Taylor Linearisation, Minimization of the Lucas/Kanade Method		solution
22/05	Assignment P03	Horn and Schunck Method	sources	solution
29/05	Assignment T04	Motion Tensor Notation, Affine Horn and Schunck, Photometric Invariants, Incremental Energy		solution
05/06	Assignment P05	Gradient Constancy and Backward Registration	sources	solution
12/06	Assignment T06	Dataterm Robustification, Gauss-Seidel with Warping		solution
19/06	Assignment T07	Smoothness Terms		solution
26/06	Assignment P08	Coarse-to-Fine Warping	sources	solution
03/07	Assignment T09	Fundamental Matrix, Stereo Reconstruction		solution
10/07	Assignment P10	Fundamental Matrix Estimation	sources	solution
17/07	Assignment T11	Variational Optic Flow with Epipolar Term		solution
24/07	Assignment P12	Mutal Information	sources	solution

A sample exam can be found below. It is designed to provide problems similar to the real exam.

Sample Exam Questions

Solution of Sample Exam Questions

There will be two closed book, written exams:

The first written exam will take place on
Tuesday, August 04, 2015, 10-12, building E1.3, lecture hall 002.

The second written exam will take place on
Tuesday, September 22, 2015, 14-16, building E1.3, lecture hall 002.

In order to qualify for the exams, attendance of the tutorials is mandatory. You will not be allowed to take part in the exams if you miss more than two tutorials. In case of qualification, you are allowed to take part in both exams. The final grade will be the best grade out of the two exams.

Please remember that you have to register online for the lecture/exam in the HISPOS system of the Saarland University

---

The results of the first written exam can be found here.

Each student who has participated in the first written exam had the opportunity to inspect his/her graded solutions in room 4.10, building E1.7, on Friday, August 14, 2015, between 10 a.m. and 12 a.m.

---

The results of the second written exam can be found here.

Each student who has participated in the second written exam has the opportunity to inspect his/her graded solutions in room 4.10, building E1.7, on Friday, October 9, 2015, between 10 a.m. and 12 a.m.

There is no specific book that covers the complete content of this class. Many lectures will be based on articles from journals and conferences. However, the following three books cover most topics:

1. Optic Flow
   A. Bruhn: Variational Optic Flow Computation: Accurate Modeling and Efficient Numerics.
   Ph.D. Thesis, 2006. Available from /bruhn/PhDThesis.pdf
   
   
2. Stereo Reconstruction
   L. Valgaerts: Variational 3D Reconstruction from Stereo Image Pairs and Stereo Sequences
   Ph.D. Thesis, 2011. Available from http://www.mpi-inf.mpg.de/~valgaerts/pubs/valgaerts_phdthesis.pdf
   
   
3. Medical Image Registration
   J. Modersitzki: Numerical Methods for Image Registration. Oxford Press, 2003.
   

---