People Teaching Publications Contact Applications

Welcome

The Neural Data Science group led by Alexander Ecker works at the interface of machine learning and computational neuroscience. We develop new methods and algorithms to make sense of large-scale neuroscience data. Moreover, we work on novel approaches to computer vision based on insights we gain from biological vision.

Currently, the Neural Data Science Group has 2 PhD students and 2 postdocs.

Data Science is a highly collaborative endeavour. We work closely with a number of labs in Göttingen, Germany and the United States: Fabian Sinz (Uni Göttingen), Andreas Tolias (Baylor College of Medicine; Houston, TX, USA), Thomas Euler (University of Tübingen), Tim Gollisch (University Medical Center, Göttingen), Christoph Kleinn (Uni Göttingen), Eberhard Bodenschatz (MPI for Dynamics and Self-Organization, Göttingen), Viola Priesemann (MPI for Dynamics and Self-Organization, Göttingen), Michael Wibral (Uni Göttingen), Michael Wilczek (Uni Bayreuth).

Workshop “Current Topics in Neural Data Science”

From August 16th to August 19th 2022, the neural data science group organized the “Current Topics in Neural Data Science” workshop.

Teaching

Summer term 2023

Practical course on applying deep learning for image generation.

Alexander Ecker and Timo Lüddecke

Winter term 2022/2023

Seminar where recent deep learning papers are presented and discussed.

Alexander Ecker, Laura Hansel, Richard Vogg, Polina Turishcheva and Timo Lüddecke

Summer term 2022

Seminar where recent computational neuroscience papers are presented and discussed.

Alexander Ecker, Laura Pede, Michaela Vystrčilová, Suhas Shrinivasan

Practical course on applying deep learning for image generation.

Alexander Ecker and Timo Lüddecke

Introduction to Machine Learning

Alexander Ecker

Bachelor’s and Master’s theses

General requirements

We expect prospective students to have substantial knowledge in machine learning, its mathematical foundations and Python programming. We therefore strongly recommend that students interested in doing their thesis in our lab should take our courses on Machine Learning, Deep Learning and took the Fachpraktikum Data Science. Exceptions are possible if well motivated.

Further recommended lectures are:

M.Inf.2201: Probabilistic Machine Learning (by Fabian Sinz)
B.Inf.1231: Infrastructures for Data Science (by Philipp Wieder)
M.WIWI-QMW.0002: Advanced Statistical Inference (by Elisabeth Bergherr)

Please note, our thesis supervision capacity is limited and we receive more thesis inquiries than we are able supervise. Therefore, we have to select candidates. If you are interested, please write an email with the subject “Master’s thesis” or “Bachelor’s thesis” containing one to three sentences about what you would like to work on and your study record to the supervisor stated below.

We will get back to you within a few days. Otherwise, do not hesitate to remind us :).

Thesis offers

Exploiting the Tree Structure of Neurons: Directed Tree Neural Networks

Supervisor: Martin Ritzert

Embedding Unbranched Segments of Neuronal Dendrites for Neuron Clustering

Supervisor: Martin Ritzert

Extend GraphDINO to incorporate modern Graph Neural Networks as their inner model

Supervisor: Martin Ritzert

Protein Structure Analysis

Supervisor: Alexander Ecker

Research

M. A. Weis, K. Chitta, Y. Sharma, W. Brendel, M. Bethge, A. Geiger, A. S. Ecker

Benchmarking Unsupervised Object Representations for Video Sequences

Journal of Machine Learning Research, 2021

@article{Weis2021, title: {Benchmarking Unsupervised Object Representations for Video Sequences}, author: {M. A. Weis, K. Chitta, Y. Sharma, W. Brendel, M. Bethge, A. Geiger, A. S. Ecker}, year: {2021}, journal: {Journal of Machine Learning Research}, }

code

.bib

K. Lurz, M. Bashiri, K. Willeke, A. Jagadish, E. Wang, E. Y. Walker, S. A. Cadena, T. Muhammad, E. Cobos, A. S. Tolias, A. S. Ecker, F. H. Sinz

Generalization in data-driven models of primary visual cortex

International Conference on Learning Representations, 2021

@inproceedings{lurz2021generalization, title: {Generalization in data-driven models of primary visual cortex}, author: {K. Lurz, M. Bashiri, K. Willeke, A. Jagadish, E. Wang, E. Y. Walker, S. A. Cadena, T. Muhammad, E. Cobos, A. S. Tolias, A. S. Ecker, F. H. Sinz}, year: {2021}, booktitle: {International Conference on Learning Representations}, }

code

.bib

M. Burg, S. Cadena, G. Denfield, E. Walker, A. Tolias, M. Bethge, A. Ecker

Learning divisive normalization in primary visual cortex

PLOS Computational Biology, 2021

show abstract

Divisive normalization (DN) is a prominent computational building block in the brain that has been proposed as a canonical cortical operation. Numerous experimental studies have verified its importance for capturing nonlinear neural response properties to simple, artificial stimuli, and computational studies suggest that DN is also an important component for processing natural stimuli. However, we lack quantitative models of DN that are directly informed by measurements of spiking responses in the brain and applicable to arbitrary stimuli. Here, we propose a DN model that is applicable to arbitrary input images. We test its ability to predict how neurons in macaque primary visual cortex (V1) respond to natural images, with a focus on nonlinear response properties within the classical receptive field. Our model consists of one layer of subunits followed by learned orientation-specific DN. It outperforms linear-nonlinear and wavelet-based feature representations and makes a significant step towards the performance of state-of-the-art convolutional neural network (CNN) models. Unlike deep CNNs, our compact DN model offers a direct interpretation of the nature of normalization. By inspecting the learned normalization pool of our model, we gained insights into a long-standing question about the tuning properties of DN that update the current textbook description: we found that within the receptive field oriented features were normalized preferentially by features with similar orientation rather than non-specifically as currently assumed.

@article{burg_2021_learning_divisive_normalization, title: {Learning divisive normalization in primary visual cortex}, author: {M. Burg, S. Cadena, G. Denfield, E. Walker, A. Tolias, M. Bethge, A. Ecker}, year: {2021}, journal: {PLOS Computational Biology}, }

code

.bib

T. Lüddecke, A. S. Ecker

Prompt-Based Multi-Modal Image Segmentation

arXiv preprint arXiv:2112.10003, 2021

show abstract

Image segmentation is usually addressed by training a model for a fixed set of object classes. Incorporating additional classes or more complex queries later is expensive as it requires re-training the model on a dataset that encompasses these expressions. Here we propose a system that can generate image segmentations based on arbitrary prompts at test time. A prompt can be either a text or an image. This approach enables us to create a unified model (trained once) for three common segmentation tasks, which come with distinct challenges: referring expression segmentation, zero-shot segmentation and one-shot segmentation. We build upon the CLIP model as a backbone which we extend with a transformer-based decoder that enables dense prediction. After training on an extended version of the PhraseCut dataset, our system generates a binary segmentation map for an image based on a free-text prompt or on an additional image expressing the query. Different variants of the latter image-based prompts are analyzed in detail. This novel hybrid input allows for dynamic adaptation not only to the three segmentation tasks mentioned above, but to any binary segmentation task where a text or image query can be formulated. Finally, we find our system to adapt well to generalized queries involving affordances or properties. Source code: https://eckerlab.org/code/clipseg

@article{luddecke2021prompt, title: {Prompt-Based Multi-Modal Image Segmentation}, author: {T. Lüddecke, A. S. Ecker}, year: {2021}, journal: {arXiv preprint arXiv:2112.10003}, }

code

.bib

M. A. Weis, L. Pede, T. Lüddecke, A. S. Ecker

Self-supervised Representation Learning of Neuronal Morphologies

arXiv, 2021

show abstract

Understanding the diversity of cell types and their function in the brain is one of the key challenges in neuroscience. The advent of large-scale datasets has given rise to the need of unbiased and quantitative approaches to cell type classification. We present GraphDINO, a purely data-driven approach to learning a low dimensional representation of the 3D morphology of neurons. GraphDINO is a novel graph representation learning method for spatial graphs utilizing self-supervised learning on transformer models. It smoothly interpolates between attention-based global interaction between nodes and classic graph convolutional processing. We show that this method is able to yield morphological cell type clustering that is comparable to manual feature-based classification and shows a good correspondence to expert-labeled cell types in two different species and cortical areas. Our method is applicable beyond neuroscience in settings where samples in a dataset are graphs and graph-level embeddings are desired.

@article{weis2021selfsupervised, title: {Self-supervised Representation Learning of Neuronal Morphologies}, author: {M. A. Weis, L. Pede, T. Lüddecke, A. S. Ecker}, year: {2021}, journal: {arXiv}, }

code

.bib

D. Kobak, Y. Bernaerts, M. Weis, F. Scala, A. Tolias, P. Berens

Sparse reduced-rank regression for exploratory visualisation of paired multivariate data

Journal of the Royal Statistical Society: Series C (Applied Statistics), 2021

@article{weis2021sparse, title: {Sparse reduced-rank regression for exploratory visualisation of paired multivariate data}, author: {D. Kobak, Y. Bernaerts, M. Weis, F. Scala, A. Tolias, P. Berens}, year: {2021}, journal: {Journal of the Royal Statistical Society: Series C (Applied Statistics)}, }

code

.bib

V. Benson, A. Ecker

Assessing out-of-domain generalization for robust building damage detection

NeurIPS 2020 Workshop on Artificial Intelligence for Humanitarian Assistance and Disaster Response (AI+HADR 2020), 2020

@inproceedings{benson2020assessing, title: {Assessing out-of-domain generalization for robust building damage detection}, author: {V. Benson, A. Ecker}, year: {2020}, booktitle: {NeurIPS 2020 Workshop on Artificial Intelligence for Humanitarian Assistance and Disaster Response (AI+HADR 2020)}, }

code

.bib

T. Lüddecke, A. Ecker

CNNs efficiently learn long-range dependencies

NeurIPS 2020 Workshop on Shared Visual Representations in Human & Machine Intelligence, 2020

@inproceedings{luddeckecnns, title: {CNNs efficiently learn long-range dependencies}, author: {T. Lüddecke, A. Ecker}, year: {2020}, booktitle: {NeurIPS 2020 Workshop on Shared Visual Representations in Human & Machine Intelligence}, }

.bib

T. Lüddecke, F. Wörgötter

Fine-grained action plausibility rating

Robotics and Autonomous Systems (RAS), 2020

@article{lueddecke20, title: {Fine-grained action plausibility rating}, author: {T. Lüddecke, F. Wörgötter}, year: {2020}, journal: {Robotics and Autonomous Systems (RAS)}, }

.bib

M. Rolínek, V. Musil, A. Paulus, M. Vlastelica, C. Michaelis, G. Martius

Optimizing Rank-based Metrics with Blackbox Differentiation

Computer Vision and Pattern Recognition (CVPR), 2020

@inproceedings{Rolínek2019a, title: {Optimizing Rank-based Metrics with Blackbox Differentiation}, author: {M. Rolínek, V. Musil, A. Paulus, M. Vlastelica, C. Michaelis, G. Martius}, year: {2020}, booktitle: {Computer Vision and Pattern Recognition (CVPR)}, }

.bib

I. Ustyuzhaninov, S. A. Cadena, E. Froudarakis, P. G. Fahey, E. Y. Walker, E. Cobos, J. Reimer, F. H. Sinz, A. S. Tolias, M. Bethge, A. S. Ecker

Rotation-invariant clustering of functional cell types in primary visual cortex

International Conference on Learning Representations (ICLR), 2020

@inproceedings{Ustyuzhaninov2020a, title: {Rotation-invariant clustering of functional cell types in primary visual cortex}, author: {I. Ustyuzhaninov, S. A. Cadena, E. Froudarakis, P. G. Fahey, E. Y. Walker, E. Cobos, J. Reimer, F. H. Sinz, A. S. Tolias, M. Bethge, A. S. Ecker}, year: {2020}, booktitle: {International Conference on Learning Representations (ICLR)}, }

.bib

Z. Zhao, D. Klindt, A. M. Chagas, K. P. Szatko, L. Rogerson, D. Protti, C. Behrens, D. Dalkara, T. Schubert, M. Bethge, K. Franke, P. Berens, A. S. Ecker, T. Euler

The temporal structure of the inner retina at a single glance

Scientific Reports, 2020

@article{Z*2020a, title: {The temporal structure of the inner retina at a single glance}, author: {Z. Zhao, D. Klindt, A. M. Chagas, K. P. Szatko, L. Rogerson, D. Protti, C. Behrens, D. Dalkara, T. Schubert, M. Bethge, K. Franke, P. Berens, A. S. Ecker, T. Euler}, year: {2020}, journal: {Scientific Reports}, }

.bib

A. S. Ecker, F. H. Sinz, E. Froudarakis, P. G. Fahey, S. A. Cadena, E. Y. Walker, E. Cobos, J. Reimer, A. S. Tolias, M. Bethge

A rotation-equivariant convolutional neural network model of primary visual cortex

International Conference on Learning Representations (ICLR), 2019

@inproceedings{ecker_2019, title: {A rotation-equivariant convolutional neural network model of primary visual cortex}, author: {A. S. Ecker, F. H. Sinz, E. Froudarakis, P. G. Fahey, S. A. Cadena, E. Y. Walker, E. Cobos, J. Reimer, A. S. Tolias, M. Bethge}, year: {2019}, journal: {International Conference on Learning Representations (ICLR)}, }

code

.bib

C. Michaelis, B. Mitzkus, R. Geirhos, E. Rusak, O. Bringmann, A. S. Ecker, M. Bethge, W. Brendel

Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming

Machine Learning for Autonomous Driving Workshop, NeurIPS 2019, 2019

@inproceedings{michaelis2019dragon, title: {Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming}, author: {C. Michaelis, B. Mitzkus, R. Geirhos, E. Rusak, O. Bringmann, A. S. Ecker, M. Bethge, W. Brendel}, year: {2019}, booktitle: {Machine Learning for Autonomous Driving Workshop, NeurIPS 2019}, }

.bib

T. Lüddecke, T. Kulvicius, F. Wörgötter

Context-based Affordance Segmentation from 2D Images for Robot Action

Robotics and Autonomous Systems (RAS), 2019

@article{lueddecke19a, title: {Context-based Affordance Segmentation from 2D Images for Robot Action}, author: {T. Lüddecke, T. Kulvicius, F. Wörgötter}, year: {2019}, journal: {Robotics and Autonomous Systems (RAS)}, }

.bib

S. A. Cadena, G. H. Denfield, E. Y. Walker, L. A. Gatys, A. S. Tolias, M. Bethge, A. S. Ecker

Deep convolutional models improve predictions of macaque V1 responses to natural images

PLoS Computational Biology, 2019

@article{Cadena2019, title: {Deep convolutional models improve predictions of macaque V1 responses to natural images}, author: {S. A. Cadena, G. H. Denfield, E. Y. Walker, L. A. Gatys, A. S. Tolias, M. Bethge, A. S. Ecker}, year: {2019}, journal: {PLoS Computational Biology}, }

.bib

T. Lüddecke, A. Agostini, M. Fauth, M. Tamosiunaite, F. Wörgötter

Distributional Semantics of Objects in Visual Scenes in Comparison to Text

Artificial Intelligence, 2019

@article{lueddecke19, title: {Distributional Semantics of Objects in Visual Scenes in Comparison to Text}, author: {T. Lüddecke, A. Agostini, M. Fauth, M. Tamosiunaite, F. Wörgötter}, year: {2019}, journal: {Artificial Intelligence}, }

.bib

S. A. Cadena, F. H. Sinz, T. Muhammad, E. Froudarakis, E. Cobos, E. Y. Walker, J. Reimer, M. Bethge, A. Tolias, A. S. Ecker

How well do deep neural networks trained on object recognition characterize the mouse visual system?

NeurIPS Neuro AI Workshop, 2019

@inproceedings{Cadena2019b, title: {How well do deep neural networks trained on object recognition characterize the mouse visual system?}, author: {S. A. Cadena, F. H. Sinz, T. Muhammad, E. Froudarakis, E. Cobos, E. Y. Walker, J. Reimer, M. Bethge, A. Tolias, A. S. Ecker}, year: {2019}, journal: {NeurIPS Neuro AI Workshop}, }

.bib

R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, W. Brendel

ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

International Conference on Learning Representations (ICLR), 2019

@inproceedings{Geirhos2019a, title: {ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness}, author: {R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, W. Brendel}, year: {2019}, journal: {International Conference on Learning Representations (ICLR)}, }

.bib

E. Y. Walker, F. H. Sinz, E. Froudarakis, P. G. Fahey, T. Muhammad, A. S. Ecker, E. Cobos, J. Reimer, X. Pitkow, A. S. Tolias

Inception loops discover what excites neurons most using deep predictive models

Nature Neuroscience, 2019

@article{Walker2019, title: {Inception loops discover what excites neurons most using deep predictive models}, author: {E. Y. Walker, F. H. Sinz, E. Froudarakis, P. G. Fahey, T. Muhammad, A. S. Ecker, E. Cobos, J. Reimer, X. Pitkow, A. S. Tolias}, year: {2019}, journal: {Nature Neuroscience}, }

.bib

Neural Data Science Group
Institute of Computer Science
University of Goettingen