Design and implementation of a cubist mirror

Bachelor’s thesis: Design and implementation of a cubist mirror

The goal of style transfer [1] is to render the content of one image in the style of another (see image above). Style transfer can be performed efficiently using feedforward neural networks [2,3], enabling real-time applications. The goal of this Bachelor’s thesis is to build a “cubist mirror” – an interactive setup that films people in front of a large video screen and renders the video stream in real time in different artistic styles that can be selected interactively by the viewer. Additional features like automatic detection and segmentation of people vs. background are desirable as well. The scope of the project includes selection and configuration of the required hardware components, design and implementation of a modular and extensible software architecture as well as training and fine-tuning of neural networks for style transfer and other image recognition features.

Neural Style Transfer

Literature

[1] Gatys LA, Ecker AS, Bethge M (2016): Image style transfer using convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
[2] Johnson J, Alahi A, Fei-Fei L (2016): Perceptual losses for real-time style transfer and super-resolution. European Conference on Computer Vision.
[3] Huang X, Belongie S (2017): Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization. 2017 IEEE International Conference on Computer Vision (ICCV).

Requirements

  • Completed lecture “Deep Learning” (B.Inf.1237); ideally also practical course “Deep learning for image synthesis” (B.Inf.1833)
  • Good software engineering skills

Contact

Alexander Ecker

Neural Data Science Group
Institute of Computer Science
University of Goettingen