Self-supervised learning using view synthesis

Self-supervised learning using view synthesis

Recently, there has been great progress in synthesizing novel views on objects [1,2,3,4]. At the same time, augmentations have been found to be crucial component in self-supervised learning systems [5]. In such systems, generated views could serve as a form of augmentation which do not shift the intensity distribution of the image (other than e.g. color augmentation).

Literature

[1] Wiles et al.: SynSin: End-to-end View Synthesis from a Single Image
(http://www.robots.ox.ac.uk/~ow/synsin.html)
[2] NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
[3] https://shihmengli.github.io/3D-Photo-Inpainting
[4] https://www-users.cs.umn.edu/~jsyoon/dynamic_synth
[5] T. Chen et al.: A simple framework for contrastive learning of visual representations

Requirements

  • Good mathematical understanding (in particular statistics and linear algebra)
  • Python programming
  • Experience in machine learning

Contact

Timo L├╝ddecke

Neural Data Science Group
Institute of Computer Science
University of Goettingen