Correspondence Problems in Computer Vision

Computer Science Teaching Award (summer term 2008)

Lecturer: Pascal Peter
Office hours: Wednesday, 15:30-16:30

Summer Term 2017

Lectures (2h) with programming/theoretical exercises (2h)
(6 credit points)

Lectures: Monday, 16-18 c.t.
Lectures: building E 1.3, lecture hall 003

First lecture: Thursday, April 20, 2017, 8:30-10 (sharp).

Tutorials: 2 hours each week, Thursday 8:30-10 (sharp).
Tutorials: either building E1.3, lecture hall 003 (theory)
Tutorials: or building E1.3, CIP-pool 104 (programming)

First tutorial: Thursday, April 27, 2017 (programming)

10/10/2017:	The results of the second written exam are available here.
02/08/2017:	The results of the first written exam are available here.
28/04/2017:	Reminder: Due to a public holiday, lecture 3 has been moved to Tuesday, 02/05, 18-20, E1.3, Lecture Hall 003.
27/04/2017:	The registration is closed.
26/04/2017:	The room for the programming tutorials has changed to CIP-pool 104 in building E1.3.
20/04/2017:	The registration is open, see here.
14/03/2017:	The first lecture is shifted to Thursday, April 20, 8:30-10 (sharp), building E1.3, lecture hall 003, due to a holiday.
14/03/2017:	The course homepage is online. Important news will be posted here.

---

Synopsis – Prerequisites – Registration – Contents – Tutorials and Assignments – Exams – References

---

Example - Stereo Reconstruction

Example - Motion Estimation

Correspondence problems are a central topic in computer vision. The basic task amounts to identifying and matching corresponding features in different images/views of the same scene. Typical examples for correspondence problems are (i) the estimation of motion information from consecutive frames of an image sequence (optic flow), (ii) the reconstruction of a 3-D scene from a stereo image pair and (iii) the registration of medical image data from different image acquisition devices (e.g. CT and MRT).
The central part of this lecture is concerned with discussing the most important correspondence problems together with suitable algorithms for their solution.

Suited for students of visual computing, applied mathematics and computer science. Requires undergraduate knowledge in mathematics (e.g. ''Mathematik für Informatiker I-III'') and elementary C knowledge (for the programming assignments). Knowledge in image processing or differential equations is useful. The lectures will be given in English.

In order to participate in the lecture/exam you have to register twice:

• Firstly, you have to register online to the system of the MIA group. This gives us an estimate of the number of students attending the course and allows us to issue the certificates at the end of the semester. You could registered for this lecture between April, 20, 2017, and April 27, 2017.
• Secondly, depending on your field of study, you have to register online for the lecture/exam in the general system of Saarland University: the HISPOS system.

Participants of the course can download the lecture materials here shortly before the lecture (access is password-protected). However, be aware that these slides are only provided to support the classroom teaching, not to replace it. Additional organisational information, such as examples and explanations that may be helpful or necessary to understand the content of the course (and thus relevant for the exam), will be provided in the lectures. It is solely your responsibility - not ours - to make sure that you receive this infomation. The following table shows a preliminary list of topics that will be covered during the semester.

Date	Lecture	Topic
20/04	Lecture 01	Introduction, Overview
24/04	Lecture 02	Block Matching, Correlation Methods, Occlusion Detection, Interest Points, Feature Methods
02/05	Lecture 03	Optic Flow I: Local Differential Methods, Parametrisation Models Due to a holiday, this lecture is shifted to Tuesday, 02/05, 18-20, E1.3, Lecture Hall 003
08/05	Lecture 04	Optic Flow II: Global Differential Methods, Horn and Schunck
15/05	Lecture 05	Optic Flow III: Advanced Constancy Assumptions, Large Motion
22/05	Lecture 06	Optic Flow IV: Advanced Data and Smoothness Terms
29/05	Lecture 07	Optic Flow V: High Accuracy Methods
12/06	Lecture 08	Optic Flow VI: Advanced Numerics Due to a conference, there is no lecture on 05/06.
19/06	Lecture 09	Stereo Matching I: Projective and Epipolar Geometry
26/06	Lecture 10	Stereo Matching II: Estimation of the Fundamental Matrix
03/07	Lecture 11	Optic Flow + Stereo: Scene Flow Estimation
10/07	Lecture 12	Medical Image Registration: Mutual Information, Elastic and Curvature Based Registration, Landmarks
17/07	Lecture 13	Particle Image Velocimetry: Div-Curl-Regularisation, Incompressible Navier Stokes Prior
24/07	Lecture 14	Summary, Outlook

The weekly tutorials alternate between theory and programming. They take place in:

Building E1.3, lecture hall 003 (theory)
Building E1.3, CIP-pool 104 (programming)

Programming excercises and theoretical assignments are offered as part of the tutorials. The regular attendance of these tutorials is required for the admission to the exams. You will not be allowed to take part in the exams if you miss more than two tutorials.

The assignments,the source code needed for the programming assignments as well as example solutions for all the assignments will be provided here during the semester.

Date	Assignment	Topic	Material	Solution
27/04	Assignment P01	Block Matching	sources	solution
04/05	Assignment T02	Sub-Pixel Refinement, Taylor Linearisation, Minimization of the Lucas/Kanade Method		solution
11/05	Assignment P03	Horn and Schunck Method	sources	solution
18/05	Assignment T04	Motion Tensor Notation, Affine Horn and Schunck, Photometric Invariants, Incremental Energy		solution
24/05	Assignment P05 ( Hints )	Gradient Constancy and Backward Registration Due to a holiay, this tutorial is shifted to Tuesday, 24/05, 18-20, E1.3, CIP 104	sources	solution
01/07	Assignment T06	Dataterm Robustification, Gauss-Seidel with Warping		solution
14/06	Assignment T07	Smoothness Terms Due to a holiay, this tutorial is shifted to Wednesday, 14/06, 18-20, E1.3, Lecture Hall 001		solution
22/06	Assignment P08 ( Hints )	Coarse-to-Fine Warping	sources	solution
29/06	Assignment T09	Fundamental Matrix, Stereo Reconstruction		solution
06/07	Assignment P10 ( Hints )	Fundamental Matrix Estimation	sources	solution
13/07	Assignment T11	Variational Optic Flow with Epipolar Term		solution
20/07	Assignment P12 ( Hints )	Mutal Information	sources	solution

A sample well be published below towards the end of the semester. It is designed to provide problems similar to the real exam.

Sample Exam Questions

Solution of Sample Exam Questions

There will be two closed book, written exams:

The first written exam will take place on
Monday, July 31, 2017, 14-16, building E1.3, lecture hall 002.

The second written exam will take place on
Wednesday, October 04, 2017, 14-16, building E1.3, lecture hall 002.

In order to qualify for the exams, attendance of the tutorials is mandatory. You will not be allowed to take part in the exams if you miss more than two tutorials. In case of qualification, you are allowed to take part in both exams. The final grade will be the best grade out of the two exams.

If you are admitted (i.e. reached enough points in the tutorials), you will find your matriculation number on this list.

Please remember that you have to register online for the lecture/exam in the HISPOS system of the Saarland University

---

The results of the first written exam can be found here.
(Statistics)

Each student who has participated in the first written exam has the opportunity to inspect his/her graded solutions in room 4.10 in Bldg. E1.7 on Tuesday, August 7, 2017 in the following timeslot:
11:00 - 12:00

---

The results of the second written exam can be found here.
(Statistics)

Each student who has participated in the second written exam has the opportunity to inspect his/her graded solutions in room 4.10 in Bldg. E1.7 on Thursday, October 12, 2017 in the following timeslot:
10:15 - 11:00

There is no specific book that covers the complete content of this class. Many lectures will be based on articles from journals and conferences. However, the following three books cover most topics:

1. Optic Flow
   A. Bruhn: Variational Optic Flow Computation: Accurate Modeling and Efficient Numerics.
   Ph.D. Thesis, 2006. Available from SULB Uni Saarland
   
   
2. Stereo Reconstruction
   L. Valgaerts: Variational 3D Reconstruction from Stereo Image Pairs and Stereo Sequences
   Ph.D. Thesis, 2011. Available from http://www.mpi-inf.mpg.de/~valgaerts/pubs/valgaerts_phdthesis.pdf
   
   
3. Medical Image Registration
   J. Modersitzki: Numerical Methods for Image Registration. Oxford Press, 2003.
   

---