Current Topics in Deep Learning

In this seminar recent deep learning papers will be presented and discussed. The goal is to dive deeper into deep learning topics beyond what can be covered in a lecture and also to give an insight in recent research topics in our lab. To name some of the topics: vision transformer and other recent large-scale models, implicit shape models and multi-object tracking. Below there is a list of papers of which you can choose a paper you want to present. It is also possible to suggest another cool paper in one of the topics.

Recommended previous knowledge

• B.Inf.1236 Machine Learning
• B.Inf.1237 Deep Learning (the seminar can accompany lecture in the same term)

Examination

• Oral presentation (approx. 30 min.) and term paper (max. 5000 words)
• Examination requirements
  • Knowledge in a specific field of machine learning
  • Ability to present the acquired knowledge in a both orally and in a written report

Organization

There will be a starting session where papers are assigned to students. In the next weeks, in each weekly session, two papers will be presented (20 minutes) and subsequently discussed (10 to 20 minutes).

Our plan is to hold this seminar in presence in room IfI 2.101 on Tuesdays 12:00 - 14:00.

If you are interested in this course, please register at Stud.IP.

Schedule

Papers to be discussed

• Vision Transformer [Richard]
  • An Image is Worth 16x16 Words
  • End-to-End Object Detection with Transformers
• Recent large-scale models [Timo]
  • Learning Transferable Visual Models From Natural Language Supervision
    Blog
  • Language Models are Few-Shot Learners
    Extra
    Blog post on GPT-2
• Self-supervised learning [Timo]
  • A Simple Framework for Contrastive Learning of Visual Representations
    Blog
  • Barlow Twins: Self-Supervised Learning via Redundancy Reduction Code
• Tracking [Richard]
  • Tracking without bells and whistles
  • FairMOT
• Implicit shape models [Laura]
  • Deep Implicit Templates
  • Convolutional Occupancy Networks
  • Implicit Neural Representations
• Graph neural networks [Laura]
  • Graph Attention Networks
  • Semi-Supervised Classification with Graph Convolutional Networks