Mathematical Image Analysis Group, Saarland University

Seminar

Deep Learning for Visual Computing

Summer Term 2022

(Main) Seminar: Deep Learning for Visual Computing

Dr. Pascal Peter,
Dr. Patrick Trampert, ZF Friedrichshafen AG, AI Lab Saarbrücken
ZF AI Centre

Summer Term 2022

(Main) Seminar (2 h)

Notice for bachelor/master students of mathematics: This is a »Hauptseminar« in the sense of these study programs.

known data	reconstructed
From P. Peter: A Wasserstein GAN for Joint Learning of Inpainting and its Spatial Optimisation.		From Eykholt et al.: Robust Physical-World Attacks on Deep Learning Visual Classification.

NEWS:

14.04.2022: One planned topic was exchanged by another since the corresponding preprint did not appear in time for the start of the semester. Topics will be distributed and you will be invited to MS Teams as soon as the assignment phase of the seminar booking system is over.
06.04.2022: After careful deliberation we have decided that the seminar will be virtual. We do not want to force attendance in person during the ongoing pandemic situation and it makes it considerably easier for visiting audience from ZF to attend the seminar.

Important Dates – Description – Registration – Requirements – Introductory Meeting – Overview of Topics –

Important Dates

Introductory meeting (mandatory):
26.04.2022, 4:15 pm In this meeting, we will provide all information you need to successfully participate in the seminar and answer your questions. Topics will be distributed after the registration period is over.
Attendance is mandatory for all participants. Do not forget to register first.

Regular meetings during the summer term 2022 on Tuesdays, 4:15 pm

Description

Contents: Neural networks have significantly changed the area of image processing and computer vision, affecting both the forefront of current research and practical deployment in industry applications. Compared to early days of NN-based visual computing, not only the scope of different applications has broadened and performance has improved, but research focus has also shifted towards increased model transparency. In this seminar, we discuss current scientific publications from the area of deep learning and investigate the corresponding models with a hands-on approach that involves implementations. We focus in particular on image inpainting, where missing or damaged image parts have to be restored from known data. This seminar is offered in cooperation with the ZF Friedrichshafen AG, AI Lab Saarbrücken, a on campus presence of ZF in the Scheer Tower D5.2. This allows us to also take a look at industry applications such as autonomous driving and AI in manufacturing.

Prerequisites: This seminar is for advanced bachelor or master students in Visual Computing, Computer Science, or Mathematics who are already acquainted with basic techniques of machine learning. In particular, they must posses some prior experience of implementing neural networks in python frameworks (Tensorflow/PyTorch). Such prior knowledge is required for the practical parts of the write-up. In addition, a basic level of experience in image processing and mathematics is strongly recommended.

Language: The publications discussed in this seminar are written in English, and English is the language of presentation.

Registration

You must register for this course via the Seminar System

from March 13, 2022
until April 12, 2022.

Requirements for Successful Participation

Regular attendance: You must attend all seminar meetings, except for provable important reasons (medical certificate).

Talk: Talk duration is 30 min, plus 15 min for discussion. Please do not deviate from this time schedule. It will be performed in a virtual setting. More details and a best practice guide will be made available after the introductory meeting.

Write-up: The write-up has to be handed at the end of the lecture period. The write-up has to take the form of a Jupyter Notebook that summarises your talk and provides practical examples for the content of the paper that was assigned to you. Which parts of the corresponding publications are to be implemented will be negotiated in the mandatory meeting with your advisor. A detailed guide for the write-up will be available towards the start of the semester. Do not forget to hand in your write-up: Participants who do not submit a write-up cannot obtain the certificate for the seminar. The write-up must be e-mailed to your advisor.

Plagiarism: Adhere to the standards of scientific referencing and avoid plagiarism: Quotations and copied material (such as images) must be clearly marked as such, and a bibliography is required. Otherwise the seminar counts as failed.

Mandatory consultation: Talk preparation has to be presented to your seminar supervisor no later than one week before the talk is given. It is your responsibility to approach your advisor timely and make your appointment.

No-shows: No-shows are unfair to your fellow students: Some talks are based on previous talks, and your seminar place might have prevented the participation of another student. Thus, in case you do not appear to your scheduled talk (except for reasons beyond your control), we reserve the right to exclude you from future seminars of our group.

Participation in discussions: The discussions after the presentations are a vital part of this seminar. This means that the audience (i.e. all participants) poses questions and tries to find positive and negative aspects of the proposed idea. This participation is part of your final grade.

Being on time: All participants have to be in the seminar call on time. Participants that turn out to be regularly late must expect a negative influence on their grade.

Introductory Meeting

Here you will find the slides from the introductory meeting. They will contain important information for preparing a good talk.

Topics

We plan to discuss the following research papers.

No.	Date	Speaker	Advisor	Topic
Classical Neural Network Inpainting
1	31.05.	Luisa Danalachi	Peter	J. Xie, L. Xu, E. Chen: Image Denoising and Inpainting with Deep Neural Networks. Advances in Neural Information Processing Systems 25 (NIPS), 2012.
2	31.05.	Bartlomiej Pogodzinski	Peter	C. Yang, X. Lu, Z. Lin, E. Shechtman, O. Wang, and H. Li High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis. Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Innovations for Deep Inpainting
3	07.06.	Eyad Alshami	Peter	J. SJ. Ren, L. Xu, Q. Yan, W. Sun: Shepard Convolutional Neural Networks. Proceedings of the 28th International Conference on Neural Information Processing Systems, Volume 1, December 2015, p. 901–909.
4	07.06.	Harsh Agarwal	Peter	D. Ulyanov, A. Vedaldi, and V. Lempitsky Deep Image Prior. International Journal of Computer Vision, Issue 128, p. 1867-1888, 2020.
Generative Adversarial Neural Networks
5	14.06.	Yudian Zheng	Peter	M. Arjovsky, S. Chintala, L. Bottou: Wasserstein generative adversarial networks. International Conference on Machine Learning (ICML), 2017.
6	14.06.	Abhijith Srinivas Bidaralli	Peter	D. Vašata, T. Halama, M. Friedjungová: Image inpainting using Wasserstein generative adversarial imputation network. Artificial Neural Networks and Machine Learning (ICANN), 2021.
Deep Spatial Optimisation
7	21.06.	Suraj Sudhakar	Peter	T. Alt, P. Peter, J. Weickert: Learning Sparse Masks for Diffusion-based Image Inpainting. arXiv:2110.02636 [eess.IV], January 2022.
8	21.06.	Rahul Nittala	Peter	P. Peter: A Wasserstein GAN for Joint Learning of Inpainting and its Spatial Optimisation. arXiv:2202.05623 [eess.IV], February 2022.
Industry Applications
9	28.06.	Xianghui Xie	Trampert	P. Trampert, D. Rubinstein, F. Boughorbel, C. Schlinkmann, M. Luschkova, P. Slusallek, T. Dahmen, and S. Sandfeld: Deep Neural Networks for Analysis of Microscopy Images—Synthetic Data Generation and Adaptive Sampling. Crystals Special Issue: Applications of Machine Learning to the Study of Crystalline Materials, Vol. 11, No. 3, March 2021.
10	28.06.	Mahnoor Shahid	Trampert	P. Perera and V. M. Patel: Learning Deep Features for One-Class Classification. IEEE Transactions on Image Processing, November 2019.
11	05.07.	Kaustubh Sakhalkar	Trampert	K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, A. Prakash, T. Kohno, D. Song: Robust Physical-World Attacks on Deep Learning Visual Classification. Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
12	05.07.	Piyush Pimplikar	Trampert	A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, A. Alahi: Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks. Conference on Computer Vision and Pattern Recognition (CVPR), 2018.