Start
Neural Data Science
People Teaching Publications Contact Applications

Welcome

The Neural Data Science group led by Alexander Ecker works at the interface of machine learning and computational neuroscience. We develop new methods and algorithms to make sense of large-scale neuroscience data. Moreover, we work on novel approaches to computer vision based on insights we gain from biological vision.

Currently, the Neural Data Science Group has 14 PhD students and 2 postdocs.

Data Science is a highly collaborative endeavour. We work closely with a number of labs in Göttingen, Germany and the United States:

Teaching

Winter term 2025/2026

Current Topics in Deep Learning

Seminar where recent deep learning papers are presented and discussed.

Alexander Ecker, Nina Nellen, Felix Benjamin Müller

Summer term 2025

Deep Learning for Image Synthesis

Practical course on applying deep learning for image generation.

Alexander Ecker and Timo Lüddecke
Lecture Machine Learning

Introduction to Machine Learning

Alexander Ecker

Winter term 2024/2025

Current Topics in Deep Learning (Foundation Models: From Deep Learning to Neuroscience)

Seminar where recent deep learning papers are presented and discussed.

Alexander Ecker, Michaela Vystrčilová, Felix Benjamin Müller
Lecture Graph Machine Learning

Introduction to Graph Machine Learning

Martin Ritzert, Alexander Ecker, and Felix Müller

Bachelor’s and Master’s theses

General requirements

If you’re interested in joining our lab for a thesis, have a look at this document describing how we work and what we expect from you.

We expect prospective students to have substantial knowledge in machine learning, its mathematical foundations and Python programming. We therefore expect students interested in doing their thesis in our lab to take our courses on Machine Learning and Deep Learning for Computer Vision unless they have acquired equivalent knowledge otherwise. For Bachelor’s students, we also recommend the Practical Course Data Science.

Further recommended courses are:

Please note, our thesis supervision capacity is limited and we receive more thesis inquiries than we are able supervise. Therefore, we have to select candidates. If you are interested, please write an email with the subject “Master’s thesis” or “Bachelor’s thesis” containing one to three sentences about what you would like to work on and your study record to the supervisor stated below.

We will get back to you within a few days. Otherwise, do not hesitate to remind us :).

Thesis offers

3D Reconstruction of a Scarred Left Ventricle
Segmentation of Left Ventricle and Scar Tissue from MRI and Mapping onto a 3D Model
Supervisor: Ina Braun
Algorithmic Association Methods for Multi-Object Tracking
Using classical graph algorithms to associate objects over time
Supervisor: Jan Frederik Meier
Analysis of Muscle Fiber Structure in Cardiac Tissue
Comparison of machine learning and physics based methods
Supervisor: Ina Braun
Approach 2D Multi-Object Tracking in 3D
Using depth estimation to resolve ambigious scenarios
Supervisor: Jan Frederik Meier
Automatic Detection, Segmentation and Classification of Lemur Vocalizations
Supervisor: Dr. Kaja Wierucka (external)
Comparing Visual Models under Human-Like Foveation Constraints
investigate foveated exploration of images
Supervisor: Valentin Hassler
Create benchmark dataset for hierarchical clustering
Create embeddings in hyperbolic space to have a dataset with innate unarguable hierarchy
Supervisor: Polina Turishcheva & Chase van de Geijn
Embeddings for neurons function and how they relate to morphology and cell types
Improve functional neuronal clustering
Supervisor: Polina Turishcheva
Few-Shot Object Detection for the Primate Domain
Adapt zero-shot detectors with few labeled images
Supervisor: Jan Frederik Meier
Implicit learning for neuronal prepresentation
Improve functional neuronal clustering
Supervisor: Polina Turishcheva
Mamba-inspired model for neuron-to-neuron architecture
Improve functional neuronal clustering
Supervisor: Polina Turishcheva
Model Neurons Interactions in time and between each other
Adjust readouts for neuroscience vision models to consider time and neurons interactions
Supervisor: Polina Turishcheva
Multiplexed traveling waves for encoding sequence position in State-Space models
Traveling Waves in Mamba
Supervisor: Chase van de Geijn
Probabilistic hierarchical clustering
Merge t-mixture models overclustercluster and merge method with hierarchical mixtures of Gaussians
Supervisor: Polina Turishcheva

Research

2025

C. v. d. Geijn, T. L{"u}ddecke, P. Turishcheva, A. S. Ecker
A Circular Argument: Does Ro{PE} need to be Equivariant for Vision?
The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025
@inproceedings{geijn2025a, title= {A Circular Argument: Does Ro{PE} need to be Equivariant for Vision?}, author= {Chase van de Geijn and Timo L{\"u}ddecke and Polina Turishcheva and Alexander S. Ecker}, year= {2025}, booktitle= {The Thirty-ninth Annual Conference on Neural Information Processing Systems}, }
.bib
M. Weis, S. Papadopoulos, L. Hansel, T. Lüddecke, B. Celii, P. Fahey, E. Wang, J. Bae, A. Bodor, D. Brittain, J. Buchanan, D. Bumbarger, M. Castro, F. Collman, N. da Costa, S. Dorkenwald, L. Elabbady, A. Halageri, Z. Jia, C. Jordan, D. Kapner, N. Kemnitz, S. Kinn, K. Lee, K. Li, R. Lu, T. Macrina, G. Mahalingam, E. Mitchell, S. Mondal, S. Mu, B. Nehoran, S. Popovych, R. Reid, C. Schneider-Mizell, H. Seung, W. Silversmith, M. Takeno, R. Torres, N. Turner, W. Wong, J. Wu, W. Yin, S. Yu, J. Reimer, P. Berens, A. Tolias, A. Ecker
An unsupervised map of excitatory neurons' dendritic morphology in the mouse visual cortex
Nature Communications, 2025
show abstract
Neurons in the neocortex exhibit astonishing morphological diversity, which is critical for properly wiring neural circuits and giving neurons their functional properties. However, the organizational principles underlying this morphological diversity remain an open question. Here, we took a data-driven approach using graph-based machine learning methods to obtain a low-dimensional morphological “bar code” describing more than 30,000 excitatory neurons in mouse visual areas V1, AL, and RL that were reconstructed from the millimeter scale MICrONS serial-section electron microscopy volume. Contrary to previous classifications into discrete morphological types (m-types), our data-driven approach suggests that the morphological landscape of cortical excitatory neurons is better described as a continuum, with a few notable exceptions in layers 5 and 6. Dendritic morphologies in layers 2–3 exhibited a trend towards a decreasing width of the dendritic arbor and a smaller tuft with increasing cortical depth. Inter-area differences were most evident in layer 4, where V1 contained more atufted neurons than higher visual areas. Moreover, we discovered neurons in V1 on the border to layer 5, which avoided deeper layers with their dendrites. In summary, we suggest that excitatory neurons’ morphological diversity is better understood by considering axes of variation than using distinct m-types.
@article{weis2025unsupervised, title= {An unsupervised map of excitatory neurons' dendritic morphology in the mouse visual cortex}, author= {Weis, Marissa A. and Papadopoulos, Stelios and Hansel, Laura and Lüddecke, Timo and Celii, Brendan and Fahey, Paul G. and Wang, Eric Y. and Bae, J. Alexander and Bodor, Agnes L. and Brittain, Derrick and Buchanan, JoAnn and Bumbarger, Daniel J. and Castro, Manuel A. and Collman, Forrest and da Costa, Nuno Maçarico and Dorkenwald, Sven and Elabbady, Leila and Halageri, Akhilesh and Jia, Zhen and Jordan, Chris and Kapner, Dan and Kemnitz, Nico and Kinn, Sam and Lee, Kisuk and Li, Kai and Lu, Ran and Macrina, Thomas and Mahalingam, Gayathri and Mitchell, Eric and Mondal, Shanka Subhra and Mu, Shang and Nehoran, Barak and Popovych, Sergiy and Reid, R. Clay and Schneider-Mizell, Casey M. and Seung, H. Sebastian and Silversmith, William and Takeno, Marc and Torres, Russel and Turner, Nicholas L. and Wong, William and Wu, Jingpeng and Yin, Wenjing and Yu, Szi-chieh and Reimer, Jacob and Berens, Philipp and Tolias, Andreas S. and Ecker, Alexander S.}, year= {2025}, journal= {Nature Communications}, }
code
.bib
R. Vogg, T. Lüddecke, J. Henrich, S. Dey, M. Nuske, V. Hassler, D. Murphy, J. Fischer, J. Ostner, O. Schülke, P. Kappeler, C. Fichtel, A. Gail, S. Treue, H. Scherberger, F. Wörgötter, A. Ecker
Computer vision for primate behavior analysis in the wild
Nature Methods, 2025
show abstract
Advances in computer vision and increasingly widespread video-based behavioral monitoring are currently transforming how we study animal behavior. However, there is still a gap between the prospects and practical application, especially in videos from the wild. In this Perspective, we aim to present the capabilities of current methods for behavioral analysis, while at the same time highlighting unsolved computer vision problems that are relevant to the study of animal behavior. We survey state-of-the-art methods for computer vision problems relevant to the video-based study of individualized animal behavior, including object detection, multi-animal tracking, individual identification and (inter)action understanding. We then review methods for effort-efficient learning, one of the challenges from a practical perspective. In our outlook on the emerging field of computer vision for animal behavior, we argue that the field should develop approaches to unify detection, tracking, identification and (inter)action understanding in a single, video-based framework.
@article{vogg2025computer, title= {Computer vision for primate behavior analysis in the wild}, author= {Vogg, Richard and Lüddecke, Timo and Henrich, Jonathan and Dey, Sharmita and Nuske, Matthias and Hassler, Valentin and Murphy, Derek and Fischer, Julia and Ostner, Julia and Schülke, Oliver and Kappeler, Peter M and Fichtel, Claudia and Gail, Alexander and Treue, Stefan and Scherberger, Hansjörg and Wörgötter, Florentin and Ecker, Alexander S}, year= {2025}, journal= {Nature Methods}, }
.bib
F. B. Müller, T. Lueddecke, R. Vogg, A. S. Ecker
Domain-Adaptive Pretraining Improves Primate Behavior Recognition
Oral @ CVPR 2025 CV4Animals Workshop, 2025
@article{mueller2025domain, title= {Domain-Adaptive Pretraining Improves Primate Behavior Recognition}, author= {Felix B. Müller and Timo Lueddecke and Richard Vogg and Alexander S. Ecker}, year= {2025}, booktitle= {Oral @ CVPR 2025 CV4Animals Workshop}, }
code
.bib
A. Paliwal, O. Schlenczek, B. Thiede, M. S. Pereira, K. Stieger, E. Bodenschatz, G. Bagheri, A. Ecker
FLASHµ: Fast Localizing And Sizing of Holographic Microparticles
ArXiv Preprint, 2025
@misc{paliwal2025flashmufastlocalizingsizing, title= {FLASHµ: Fast Localizing And Sizing of Holographic Microparticles}, author= {Ayush Paliwal and Oliver Schlenczek and Birte Thiede and Manuel Santos Pereira and Katja Stieger and Eberhard Bodenschatz and Gholamhossein Bagheri and Alexander Ecker}, year= {2025}, journal= {ArXiv Preprint}, }
code
.bib
E. Wang, P. Fahey, K. Ponder, Z. Ding, A. Change, T. Muhammad, S. Patel, Z. Ding, D. Tran, J. Fu, others
Foundation model of neural activity predicts response to new stimulus types
Nature, 2025
@article{wang2025foundation, title= {Foundation model of neural activity predicts response to new stimulus types}, author= {Wang, Eric Y and Fahey, Paul G and Ponder, Kayla and Ding, Zhuokun and Change, Andersen and Muhammad, Taliah and Patel, Saumil and Ding, Zhiwei and Tran, Dat T and Fu, Jiakun and others}, year= {2025}, journal= {Nature}, }
code
.bib
M. Ritzert, P. Turishcheva, L. Hansel, P. Wollenhaupt, M. A. Weis, A. S. Ecker
Hierarchical clustering with maximum density paths and mixture models
ArXiv Preprint, 2025
@misc{ritzert2025hierarchicalclusteringmaximumdensity, title= {Hierarchical clustering with maximum density paths and mixture models}, author= {Martin Ritzert and Polina Turishcheva and Laura Hansel and Paul Wollenhaupt and Marissa A. Weis and Alexander S. Ecker}, year= {2025}, booktitle= {ArXiv Preprint}, }
code
.bib
N. S. Nellen, P. Turishcheva, M. Vystrčilová, S. Sridhar, T. Gollisch, A. S. Tolias, A. S. Ecker
Learning to cluster neuronal function
NeurIPS, 2025
@inproceedings{nellen2025learningclusterneuronalfunction, title= {Learning to cluster neuronal function}, author= {Nina S. Nellen and Polina Turishcheva and Michaela Vystrčilová and Shashwat Sridhar and Tim Gollisch and Andreas S. Tolias and Alexander S. Ecker}, year= {2025}, booktitle= {NeurIPS}, }
code
.bib
J. van Delden, J. Schultz, S. Rothe, C. Libner, S. Langer, T. Lüddecke
Minimizing Structural Vibrations via Guided Flow Matching Design Optimization
ArXiv Preprint, 2025
@misc{delden2025minimizing, title= {Minimizing Structural Vibrations via Guided Flow Matching Design Optimization}, author= {van Delden, Jan and Schultz, Julius and Rothe, Sebastian and Libner, Christian and Langer, Sabine C and Lüddecke, Timo}, year= {2025}, booktitle= {ArXiv Preprint}, }
code
.bib
F. Schmidt, P. Turishcheva, S. Shrinivasan, F. H. Sinz
Modeling Dynamic Neural Activity by combining Naturalistic Video Stimuli and Stimulus-independent Latent Factors
NeurIPS, 2025
@inproceedings{schmidt2025latentmodeling, title= {Modeling Dynamic Neural Activity by combining Naturalistic Video Stimuli and Stimulus-independent Latent Factors}, author= {Finn Schmidt and Polina Turishcheva and Suhas Shrinivasan and Fabian H. Sinz}, year= {2025}, booktitle= {NeurIPS}, }
code
.bib
A. Schneider, V. Neuhaus, D. Ehrlich, A. Makkeh, A. Ecker, V. Priesemann, M. Wibral
What should a neuron aim for? Designing local objective functions based on information theory
Oral @ International Conference on Learning Representations, 2025
show abstract
In modern deep neural networks, the learning dynamics of individual neurons are often obscure, as the networks are trained via global optimization. Conversely, biological systems build on self-organized, local learning, achieving robustness and efficiency with limited global information. Here, we show how self-organization between individual artificial neurons can be achieved by designing abstract bio-inspired local learning goals. These goals are parameterized using a recent extension of information theory, Partial Information Decomposition (PID), which decomposes the information that a set of information sources holds about an outcome into unique, redundant and synergistic contributions. Our framework enables neurons to locally shape the integration of information from various input classes, i.e., feedforward, feedback, and lateral, by selecting which of the three inputs should contribute uniquely, redundantly or synergistically to the output. This selection is expressed as a weighted sum of PID terms, which, for a given problem, can be directly derived from intuitive reasoning or via numerical optimization, offering a window into understanding task-relevant local information processing. Achieving neuron-level interpretability while enabling strong performance using local learning, our work advances a principled information-theoretic foundation for local learning strategies.
@inproceedings{schneider2025should, title= {What should a neuron aim for? Designing local objective functions based on information theory}, author= {Schneider, Andreas C and Neuhaus, Valentin and Ehrlich, David A and Makkeh, Abdullah and Ecker, Alexander S and Priesemann, Viola and Wibral, Michael}, year= {2025}, booktitle= {Oral @ International Conference on Learning Representations}, }
code
.bib

2024

L. Höfling, K. Szatko, C. Behrens, Y. Deng, Y. Qiu, D. Klindt, Z. Jessen, G. Schwartz, M. Bethge, P. Berens, K. Franke, A. Ecker, T. Euler
A chromatic feature detector in the retina signals visual context changes
eLife, 2024
show abstract
The retina transforms patterns of light into visual feature representations supporting behaviour. These representations are distributed across various types of retinal ganglion cells (RGCs), whose spatial and temporal tuning properties have been studied extensively in many model organisms, including the mouse. However, it has been difficult to link the potentially nonlinear retinal transformations of natural visual inputs to specific ethological purposes. Here, we discover a nonlinear selectivity to chromatic contrast in an RGC type that allows the detection of changes in visual context. We trained a convolutional neural network (CNN) model on large-scale functional recordings of RGC responses to natural mouse movies, and then used this model to search in silico for stimuli that maximally excite distinct types of RGCs. This procedure predicted centre colour opponency in transient suppressed-by-contrast (tSbC) RGCs, a cell type whose function is being debated. We confirmed experimentally that these cells indeed responded very selectively to Green-OFF, UV-ON contrasts. This type of chromatic contrast was characteristic of transitions from ground to sky in the visual scene, as might be elicited by head or eye movements across the horizon. Because tSbC cells performed best among all RGC types at reliably detecting these transitions, we suggest a role for this RGC type in providing contextual information (i.e. sky or ground) necessary for the selection of appropriate behavioural responses to other stimuli, such as looming objects. Our work showcases how a combination of experiments with natural stimuli and computational modelling allows discovering novel types of stimulus selectivity and identifying their potential ethological relevance.
@article{hoefling2024chromatic, title= {A chromatic feature detector in the retina signals visual context changes}, author= {Höfling, Larissa and Szatko, Klaudia P and Behrens, Christian and Deng, Yuyao and Qiu, Yongrong and Klindt, David Alexander and Jessen, Zachary and Schwartz, Gregory W and Bethge, Matthias and Berens, Philipp and Franke, Katrin and Ecker, Alexander S and Euler, Thomas}, year= {2024}, journal= {eLife}, }
code
.bib
M. Vystrčilová, S. Sridhar, M. Burg, T. Gollisch, A. Ecker
Convolutional neural network models of the primate retina reveal adaptation to natural stimulus statistics
bioRxiv, 2024
show abstract
The diverse nature of visual environments demands that the retina, the first stage of the visual system, encodes a vast range of stimuli with various statistics. The retina adapts its computations to some specific features of the input, such as brightness, contrast or motion. However, it is less clear whether it also adapts to the statistics of natural scenes compared to white noise, the latter of which is often used to infer models of retinal computation. To address this question, we analyzed neural activity of retinal ganglion cells (RGCs) in response to both white noise and naturalistic movie stimuli. We performed a systematic comparative analysis of traditional linear-nonlinear (LN) and recent convolutional neural network (CNN) models and tested their generalization across stimulus domains. We found that no model type trained on one stimulus ensemble was able to accurately predict neural activity on the other, suggesting that retinal processing depends on the stimulus statistics. Under white noise stimulation, the receptive fields of the neurons were mostly lowpass, while under natural image statistics they exhibited a more pronounced surround resembling the whitening filters predicted by efficient coding. Together, these results suggest that retinal processing dynamically adapts to the stimulus statistics.
@article{vystrvcilova2024convolutional, title= {Convolutional neural network models of the primate retina reveal adaptation to natural stimulus statistics}, author= {Vystrčilová, Michaela and Sridhar, Shashwat and Burg, Max F and Gollisch, Tim and Ecker, Alexander S}, year= {2024}, journal= {bioRxiv}, }
.bib
S. Cadena, K. Willeke, K. Restivo, G. Denfield, F. Sinz, M. Bethge, A. Tolias, A. Ecker
Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks
PLOS Computational Biology, 2024
@article{cadena2024diverse, title= {Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks}, author= {Cadena, S. A. and Willeke, K. F. and Restivo, K. and Denfield, G. H.W. and Sinz, F. H. and Bethge, M. and Tolias, A. S. and Ecker, A. S.}, year= {2024}, journal= {PLOS Computational Biology}, }
code
.bib
J. van Delden, J. Schultz, C. Blech, S. Langer, T. Lüddecke
Learning to Predict Structural Vibrations
NeurIPS 2024, 2024
@article{delden2024vibrating, title= {Learning to Predict Structural Vibrations}, author= {van Delden, Jan and Schultz, Julius and Blech, Christopher and Langer, Sabine C and Lüddecke, Timo}, year= {2024}, journal= {NeurIPS 2024}, }
code
.bib
F. Müller, R. Görge, A. Bernzen, J. Pirk, M. Poretschkin
LLMs and Memorization: On Quality and Specificity of Copyright Compliance
2024 AAAI/ACM Conference on AI, Ethics, and Society, 2024
show abstract
Memorization in large language models (LLMs) is a growing concern. LLMs have been shown to easily reproduce parts of their training data, including copyrighted work. This is an important problem to solve, as it may violate existing copyright laws as well as the European AI Act. In this work, we propose a systematic analysis to quantify the extent of potential copyright infringements in LLMs using European law as an example. Unlike previous work, we evaluate instruction-finetuned models in a realistic end-user scenario. Our analysis builds on a proposed threshold of 160 characters, which we borrow from the German Copyright Service Provider Act and a fuzzy text matching algorithm to identify potentially copyright-infringing textual reproductions. The specificity of countermeasures against copyright infringement is analyzed by comparing model behavior on copyrighted and public domain data. We investigate what behaviors models show instead of producing protected text (such as refusal or hallucination) and provide a first legal assessment of these behaviors. We find that there are huge differences in copyright compliance, specificity, and appropriate refusal among popular LLMs. Alpaca, GPT 4, GPT 3.5, and Luminous perform best in our comparison, with OpenGPT-X, Alpaca, and Luminous producing a particularly low absolute number of potential copyright violations. Code will be published soon.
@article{mueller2024llmsmemorizationqualityspecificity, title= {LLMs and Memorization: On Quality and Specificity of Copyright Compliance}, author= {Müller, Felix B and Görge, Rebekka and Bernzen, Anna K and Pirk, Janna C and Poretschkin, Maximilian}, year= {2024}, journal= {2024 AAAI/ACM Conference on AI, Ethics, and Society}, }
code
.bib
F. Müller, J. Tanke, J. Gall
Massively Multi-Person 3D Human Motion Forecasting with Scene Context
ECCV 2024 Workshop and Competition on Affective Behavior Analysis in-the-wild, 2024
show abstract
Forecasting long-term 3D human motion is challenging: the stochasticity of human behavior makes it hard to generate realistic human motion from the input sequence alone. Information on the scene environment and the motion of nearby people can greatly aid the generation process. We propose a scene-aware social transformer model (SAST) to forecast long-term (10s) human motion motion. Unlike previous models, our approach can model interactions between both widely varying numbers of people and objects in a scene. We combine a temporal convolutional encoder-decoder architecture with a Transformer-based bottleneck that allows us to efficiently combine motion and scene information. We model the conditional motion distribution using denoising diffusion models. We benchmark our approach on the Humans in Kitchens dataset, which contains 1 to 16 persons and 29 to 50 objects that are visible simultaneously. Our model outperforms other approaches in terms of realism and diversity on different metrics and in a user study.
@article{mueller2024massivelymultiperson3dhuman, title= {Massively Multi-Person 3D Human Motion Forecasting with Scene Context}, author= {Müller, Felix B and Tanke, Julian and Gall, Jürgen}, year= {2024}, journal= {ECCV 2024 Workshop and Competition on Affective Behavior Analysis in-the-wild}, }
code
.bib
J. van Delden, J. Schultz, C. Blech, S. Langer, T. Lüddecke
Minimizing Structural Vibrations via Guided Diffusion Design Optimization
ICLR 2024 Workshop on AI4DifferentialEquations In Science, 2024
@inproceedings{delden2024minimizing, title= {Minimizing Structural Vibrations via Guided Diffusion Design Optimization}, author= {van Delden, Jan and Schultz, Julius and Blech, Christopher and Langer, Sabine C and Lüddecke, Timo}, year= {2024}, booktitle= {ICLR 2024 Workshop on AI4DifferentialEquations In Science}, }
code
.bib
P. Turishcheva, L. Hansel, M. Ritzert, M. A. Weis, A. S. Ecker
MNIST-Nd: a set of naturalistic datasets to benchmark clustering across dimensions
NeurIPS 2024 NeurReps Workshop (extended abstract), 2024
@misc{turishcheva2024mnistndsetnaturalisticdatasets, title= {MNIST-Nd: a set of naturalistic datasets to benchmark clustering across dimensions}, author= {Polina Turishcheva and Laura Hansel and Martin Ritzert and Marissa A. Weis and Alexander S. Ecker}, year= {2024}, journal= {NeurIPS 2024 NeurReps Workshop (extended abstract)}, }
code
.bib
F. Schmidt, S. Shrinivasan, P. Turishcheva, F. H. Sinz
Modeling dynamic neural activity by combining naturalistic video stimuli and stimulus-independent latent factors
NeurIPS 2024 NeurReps Workshop (extended abstract), 2024
@misc{schmidt2024modelingdynamicneuralactivity, title= {Modeling dynamic neural activity by combining naturalistic video stimuli and stimulus-independent latent factors}, author= {Finn Schmidt and Suhas Shrinivasan and Polina Turishcheva and Fabian H. Sinz}, year= {2024}, journal= {NeurIPS 2024 NeurReps Workshop (extended abstract)}, }
.bib
S. Sridhar, M. Vystrčilová, M. Khani, D. Karamanlis, H. Schreyer, V. Ramakrishna, S. Krueppel, S. Zapp, M. Mietsch, A. Ecker, T. Gollisch
Modeling spatial contrast sensitivity in responses of primate retinal ganglion cells to natural movies
bioRxiv, 2024
show abstract
Retinal ganglion cells, the output neurons of the vertebrate retina, often display nonlinear summation of visual signals over their receptive fields. This creates sensitivity to spatial contrast, letting the cells respond to spatially structured visual stimuli, such as a contrast-reversing grating, even when no net change in overall illumination of the receptive field occurs. Yet, computational models of ganglion cell responses are often based on linear receptive fields. Nonlinear extensions, on the other hand, such as subunit models, which separate receptive fields into smaller, nonlinearly combined subfields, are often cumbersome to fit to experimental data, in particular when natural stimuli are considered. Previous work in the salamander retina has shown that sensitivity to spatial contrast in response to flashed images can be partly captured by a model that combines signals from the mean and variance of luminance signals inside the receptive field. Here, we extend this spatial contrast model for application to spatiotemporal stimulation and explore its performance on spiking responses that we recorded from retinas of marmosets under artificial and natural movies. We show how the model can be fitted to experimental data and that it outperforms common models with linear spatial integration, in particular for parasol ganglion cells. Finally, we use the model framework to infer the cells’ spatial scale of nonlinear spatial integration and contrast sensitivity. Our work shows that the spatial contrast model provides a simple approach to capturing aspects of nonlinear spatial integration with only few free parameters, which can be used to assess the cells’ functional properties under natural stimulation and which provides a simple-to-obtain benchmark for comparison with more detailed nonlinear encoding models.
@article{sridhar2024modeling, title= {Modeling spatial contrast sensitivity in responses of primate retinal ganglion cells to natural movies}, author= {Sridhar, Shashwat and Vystrčilová, Michaela and Khani, Mohammad H and Karamanlis, Dimokratis and Schreyer, Helene M and Ramakrishna, Varsha and Krueppel, Steffen and Zapp, Soeren J and Mietsch, Matthias and Ecker, Alexander and Gollisch, Tim}, year= {2024}, journal= {bioRxiv}, }
.bib
M. F. Burg, T. Zenkel, M. Vystrčilová, J. Oesterle, L. Höfling, K. F. Willeke, J. Lause, S. Müller, P. G. Fahey, Z. Ding, K. Restivo, S. Sridhar, T. Gollisch, P. Berens, A. S. Tolias, T. Euler, M. Bethge, A. S. Ecker
Most discriminative stimuli for functional cell type clustering
The Twelfth International Conference on Learning Representations, 2024
@inproceedings{burg2024most, title= {Most discriminative stimuli for functional cell type clustering}, author= {Max F Burg and Thomas Zenkel and Michaela Vystrčilová and Jonathan Oesterle and Larissa Höfling and Konstantin Friedrich Willeke and Jan Lause and Sarah Müller and Paul G. Fahey and Zhiwei Ding and Kelli Restivo and Shashwat Sridhar and Tim Gollisch and Philipp Berens and Andreas S. Tolias and Thomas Euler and Matthias Bethge and Alexander S Ecker}, year= {2024}, booktitle= {The Twelfth International Conference on Learning Representations}, }
code
.bib
N. Wu, I. Valera, F. Sinz, A. Ecker, T. Euler, Y. Qiu
Probabilistic neural transfer function estimation with Bayesian system identification
PLOS Computational Biology, 2024
@article{wu2024probabilistic, title= {Probabilistic neural transfer function estimation with Bayesian system identification}, author= {Wu, Nan and Valera, Isabel and Sinz, Fabian and Ecker, Alexander and Euler, Thomas and Qiu, Yongrong}, year= {2024}, journal= {PLOS Computational Biology}, }
code
.bib
P. Turishcheva, M. Burg, F. H. Sinz, A. Ecker
Reproducibility of predictive networks for mouse visual cortex
NeurIPS 2024 (Spotlight), 2024
@misc{turishcheva2024reproducibilitypredictivenetworksmouse, title= {Reproducibility of predictive networks for mouse visual cortex}, author= {Polina Turishcheva and Max Burg and Fabian H. Sinz and Alexander Ecker}, year= {2024}, journal= {NeurIPS 2024 (Spotlight)}, }
code
.bib
P. Turishcheva, P. G. Fahey, M. Vystrčilová, L. Hansel, R. Froebe, K. Ponder, Y. Qiu, K. F. Willeke, M. Bashiri, R. Baikulov, Y. Zhu, L. Ma, S. Yu, T. Huang, B. M. Li, W. D. Wulf, N. Kudryashova, M. H. Hennig, N. L. Rochefort, A. Onken, E. Wang, Z. Ding, A. S. Tolias, F. H. Sinz, A. S. Ecker
Retrospective for the Dynamic Sensorium Competition for predicting large-scale mouse primary visual cortex activity from videos
NeurIPS 2024 (Dataset and Benchmarks), 2024
@misc{turishcheva2024retrospectivedynamicsensoriumcompetition, title= {Retrospective for the Dynamic Sensorium Competition for predicting large-scale mouse primary visual cortex activity from videos}, author= {Polina Turishcheva and Paul G. Fahey and Michaela Vystrčilová and Laura Hansel and Rachel Froebe and Kayla Ponder and Yongrong Qiu and Konstantin F. Willeke and Mohammad Bashiri and Ruslan Baikulov and Yu Zhu and Lei Ma and Shan Yu and Tiejun Huang and Bryan M. Li and Wolf De Wulf and Nina Kudryashova and Matthias H. Hennig and Nathalie L. Rochefort and Arno Onken and Eric Wang and Zhiwei Ding and Andreas S. Tolias and Fabian H. Sinz and Alexander S Ecker}, year= {2024}, journal= {NeurIPS 2024 (Dataset and Benchmarks)}, }
code
.bib
M. Carbone, V. Peterhans, A. Ecker, M. Wilczek
Tailor-Designed Models for the Turbulent Velocity Gradient through Normalizing Flow
Phys. Rev. Lett. (Editors' Suggestion), 2024
@article{carbone2024tailor, title= {Tailor-Designed Models for the Turbulent Velocity Gradient through Normalizing Flow}, author= {Carbone, M. and Peterhans, V. J. and Ecker, A. S. and Wilczek, M.}, year= {2024}, journal= {Phys. Rev. Lett. (Editors' Suggestion)}, }
code
.bib

2023

N. Wu, I. Valera, A. Ecker, T. Euler, Y. Qiu
Bayesian Neural System Identification with Response Variability
arXiv preprint arXiv:2308.05990, 2023
@article{wu2023bayesian, title= {Bayesian Neural System Identification with Response Variability}, author= {Wu, Nan and Valera, Isabel and Ecker, Alexander and Euler, Thomas and Qiu, Yongrong}, year= {2023}, journal= {arXiv preprint arXiv:2308.05990}, }
.bib
Z. Ding, D. Tran, K. Ponder, E. Cobos, Z. Ding, P. Fahey, E. Wang, T. Muhammad, J. Fu, S. Cadena, others
Bipartite invariance in mouse primary visual cortex
bioRxiv, 2023
@article{ding2023bipartite, title= {Bipartite invariance in mouse primary visual cortex}, author= {Ding, Zhiwei and Tran, Dat T and Ponder, Kayla and Cobos, Erick and Ding, Zhuokun and Fahey, Paul G and Wang, Eric and Muhammad, Taliah and Fu, Jiakun and Cadena, Santiago A and others}, year= {2023}, journal= {bioRxiv}, }
.bib
A. Barreto, L. Reifenrath, R. Vogg, F. Sinz, A. Mahlein
Data Augmentation for Mask-Based Leaf Segmentation of UAV-Images as a Basis to Extract Leaf-Based Phenotyping Parameters
KI-Künstliche Intelligenz, 2023
show abstract
In crop protection, disease quantification parameters such as disease incidence (DI) and disease severity (DS) are the principal indicators for decision making, aimed at ensuring the safety and productivity of crop yield. The quantification is standardized with leaf organs, defined as individual scoring units. This study focuses on identifying and segmenting individual leaves in agricultural fields using unmanned aerial vehicle (UAV), multispectral imagery of sugar beet fields, and deep instance segmentation networks (Mask R-CNN). Five strategies for achieving network robustness with limited labeled images are tested and compared, employing simple and copy-paste image augmentation techniques. The study also evaluates the impact of environmental conditions on network performance. Metrics of performance show that multispectral UAV images recorded under sunny conditions lead to a performance drop. Focusing on the practical application, we employ Mask R-CNN models in an image-processing pipeline to calculate leaf-based parameters including DS and DI. The pipeline was applied in time-series in an experimental trial with five varieties and two fungicide strategies to illustrate epidemiological development. Disease severity calculated with the model with highest Average Precision (AP) shows the strongest correlation with the same parameter assessed by experts. The time-series development of disease severity and disease incidence demonstrates the advantages of multispectral UAV-imagery in contrasting varieties for resistance, as well as the limits for disease control measurements. This study identifies key components for automatic leaf segmentation of diseased plants using UAV imagery, such as illumination and disease condition. It also provides a tool for delivering leaf-based parameters relevant to optimize crop production through automated disease quantification by imaging tools.
@article{barreto2023data, title= {Data Augmentation for Mask-Based Leaf Segmentation of UAV-Images as a Basis to Extract Leaf-Based Phenotyping Parameters}, author= {Barreto, Abel and Reifenrath, Lasse and Vogg, Richard and Sinz, Fabian and Mahlein, Anne-Katrin}, year= {2023}, journal= {KI-Künstliche Intelligenz}, }
.bib
J. Schultz, J. van Delden, C. Blech, S. Langer, T. Lüddecke
Deep learning for frequency response prediction of a multimass oscillator
PAMM, 2023
@article{schultz2023deep, title= {Deep learning for frequency response prediction of a multimass oscillator}, author= {Schultz, Julius and van Delden, Jan and Blech, Christopher and Langer, Sabine C and Lüddecke, Timo}, year= {2023}, journal= {PAMM}, }
code
.bib
K. Willeke, K. Restivo, K. Franke, A. Nix, S. Cadena, T. Shinn, C. Nealley, G. Rodriguez, S. Patel, A. Ecker, others
Deep learning-driven characterization of single cell tuning in primate visual area V4 unveils topological organization
bioRxiv, 2023
@article{willeke2023deep, title= {Deep learning-driven characterization of single cell tuning in primate visual area V4 unveils topological organization}, author= {Willeke, Konstantin F and Restivo, Kelli and Franke, Katrin and Nix, Arne F and Cadena, Santiago A and Shinn, Tori and Nealley, Cate and Rodriguez, Gabby and Patel, Saumil and Ecker, Alexander S and others}, year= {2023}, journal= {bioRxiv}, }
.bib
J. Fu, S. Shrinivasan, K. Ponder, T. Muhammad, Z. Ding, E. Wang, Z. Ding, D. Tran, P. Fahey, S. Papadopoulos, others
Pattern completion and disruption characterize contextual modulation in mouse visual cortex
bioRxiv, 2023
@article{fu2023pattern, title= {Pattern completion and disruption characterize contextual modulation in mouse visual cortex}, author= {Fu, Jiakun and Shrinivasan, Suhas and Ponder, Kayla and Muhammad, Taliah and Ding, Zhuokun and Wang, Eric and Ding, Zhiwei and Tran, Dat T and Fahey, Paul G and Papadopoulos, Stelios and others}, year= {2023}, journal= {bioRxiv}, }
.bib
M. A. Weis, L. Hansel, T. Lüddecke, A. S. Ecker
Self-Supervised Graph Representation Learning for Neuronal Morphologies
Transactions on Machine Learning Research, 2023
show abstract
Unsupervised graph representation learning has recently gained interest in several application domains such as neuroscience, where modeling the diverse morphology of cell types in the brain is one of the key challenges. It is currently unknown how many excitatory cortical cell types exist and what their defining morphological features are. Here we present GraphDINO, a purely data-driven approach to learn low-dimensional representations of 3D neuronal morphologies from unlabeled large-scale datasets. GraphDINO is a novel transformer-based representation learning method for spatially-embedded graphs. To enable self-supervised learning on transformers, we (1) developed data augmentation strategies for spatially-embedded graphs, (2) adapted the positional encoding and (3) introduced a novel attention mechanism, AC-Attention, which combines attention-based global interaction between nodes and classic graph convolutional processing. We show, in two different species and across multiple brain areas, that this method yields morphological cell type clusterings that are on par with manual feature-based classification by experts, but without using prior knowledge about the structural features of neurons. Moreover, it outperforms previous approaches on quantitative benchmarks predicting expert labels. Our method could potentially enable data-driven discovery of novel morphological features and cell types in large-scale datasets. It is applicable beyond neuroscience in settings where samples in a dataset are graphs and graph-level embeddings are desired.
@article{weis2023selfsupervised, title= {Self-Supervised Graph Representation Learning for Neuronal Morphologies}, author= {Marissa A. Weis and Laura Hansel and Timo Lüddecke and Alexander S Ecker}, year= {2023}, journal= {Transactions on Machine Learning Research}, }
code
.bib
P. Turishcheva, P. Fahey, L. Hansel, R. Froebe, K. Ponder, M. Vystrčilová, K. Willeke, M. Bashiri, E. Wang, Z. Ding, others
The Dynamic Sensorium competition for predicting large-scale mouse visual cortex activity from videos
arXiv preprint arXiv:2305.19654, 2023
show abstract
Understanding how biological visual systems process information is challenging due to the complex nonlinear relationship between neuronal responses and high-dimensional visual input. Artificial neural networks have already improved our understanding of this system by allowing computational neuroscientists to create predictive models and bridge biological and machine vision. During the Sensorium 2022 competition, we introduced benchmarks for vision models with static input. However, animals operate and excel in dynamic environments, making it crucial to study and understand how the brain functions under these conditions. Moreover, many biological theories, such as predictive coding, suggest that previous input is crucial for current input processing. Currently, there is no standardized benchmark to identify state-of-the-art dynamic models of the mouse visual system. To address this gap, we propose the Sensorium 2023 Competition with dynamic input. This includes the collection of a new large-scale dataset from the primary visual cortex of five mice, containing responses from over 38,000 neurons to over 2 hours of dynamic stimuli per neuron. Participants in the main benchmark track will compete to identify the best predictive models of neuronal responses for dynamic input. We will also host a bonus track in which submission performance will be evaluated on out-of-domain input, using withheld neuronal responses to dynamic input stimuli whose statistics differ from the training set. Both tracks will offer behavioral data along with video stimuli. As before, we will provide code, tutorials, and strong pre-trained baseline models to encourage participation. We hope this competition will continue to strengthen the accompanying Sensorium benchmarks collection as a standard tool to measure progress in large-scale neural system identification models of the entire mouse visual hierarchy and beyond.
@article{turishcheva2023dynamic, title= {The Dynamic Sensorium competition for predicting large-scale mouse visual cortex activity from videos}, author= {Turishcheva, Polina and Fahey, Paul G and Hansel, Laura and Froebe, Rachel and Ponder, Kayla and Vystrčilová, Michaela and Willeke, Konstantin F and Bashiri, Mohammad and Wang, Eric and Ding, Zhiwei and others}, year= {2023}, journal= {arXiv preprint arXiv:2305.19654}, }
code
.bib
T. Jiang, M. Freudenberg, C. Kleinn, A. Ecker, N. Nölke
The Impacts of Quality-Oriented Dataset Labeling on Tree Cover Segmentation Using U-Net: A Case Study in WorldView-3 Imagery
Remote Sensing, 2023
@article{jiang2023impacts, title= {The Impacts of Quality-Oriented Dataset Labeling on Tree Cover Segmentation Using U-Net: A Case Study in WorldView-3 Imagery}, author= {Jiang, Tao and Freudenberg, Maximilian and Kleinn, Christoph and Ecker, Alexander and Nölke, Nils}, year= {2023}, journal= {Remote Sensing}, }
.bib

2022

L. Hoefling, K. Szatko, C. Behrens, Y. Qiu, D. Klindt, Z. Jessen, G. Schwartz, M. Bethge, P. Berens, K. Franke, A. Ecker, T. Euler
A chromatic feature detector in the retina signals visual context changes
bioRxiv, 2022
@article{hoefling2022chromatic, title= {A chromatic feature detector in the retina signals visual context changes}, author= {Hoefling, Larissa and Szatko, Klaudia P and Behrens, Christian and Qiu, Yongrong and Klindt, David Alexander and Jessen, Zachary and Schwartz, Gregory S and Bethge, Matthias and Berens, Philipp and Franke, Katrin and Ecker, Alexander S and Euler, Thomas}, year= {2022}, journal= {bioRxiv}, }
.bib
M. Goldin, B. Lefebvre, S. Virgili, M. Pham Van Cang, A. Ecker, T. Mora, U. Ferrari, O. Marre
Context-dependent selectivity to natural images in the retina
Nature Communications, 2022
@article{goldin2022context, title= {Context-dependent selectivity to natural images in the retina}, author= {Goldin, Mat{\'\i}as A and Lefebvre, Baptiste and Virgili, Samuele and Pham Van Cang, Mathieu Kim and Ecker, Alexander and Mora, Thierry and Ferrari, Ulisse and Marre, Olivier}, year= {2022}, journal= {Nature Communications}, }
.bib
S. Cadena, K. Willeke, K. Restivo, G. Denfield, F. Sinz, M. Bethge, A. Tolias, A. Ecker
Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks
bioRxiv, 2022
@article{cadena2022diverse, title= {Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks}, author= {Cadena, Santiago A and Willeke, Konstantin F and Restivo, Kelli and Denfield, George and Sinz, Fabian H and Bethge, Matthias and Tolias, Andreas S and Ecker, Alexander S}, year= {2022}, journal= {bioRxiv}, }
.bib
T. Lüddecke, A. S. Ecker
Image Segmentation Using Text and Image Prompts
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
@article{lueddecke2021clipseg, title= {Image Segmentation Using Text and Image Prompts}, author= {Timo Lüddecke and Alexander S. Ecker}, year= {2022}, journal= {2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, }
code
.bib
K. F. Willeke, P. G. Fahey, M. Bashiri, L. Hansel, M. F. Burg, C. Blessing, S. A. Cadena, Z. Ding, K. Lurz, K. Ponder, T. Muhammad, S. S. Patel, A. S. Ecker, A. S. Tolias, F. H. Sinz
Retrospective on the SENSORIUM 2022 competition
PMLR, 2022
show abstract
The neural underpinning of the biological visual system is challenging to study experi- mentally, in particular as neuronal activity becomes increasingly nonlinear with respect to visual input. Artificial neural networks (ANNs) can serve a variety of goals for improving our understanding of this complex system, not only serving as predictive digital twins of sensory cortex for novel hypothesis generation in silico, but also incorporating bio-inspired architectural motifs to progressively bridge the gap between biological and machine vision. The mouse has recently emerged as a popular model system to study visual information processing, but no standardized large-scale benchmark to identify state-of-the-art mod- els of the mouse visual system has been established. To fill this gap, we proposed the SENSORIUM benchmark competition. We collected a large-scale dataset from mouse primary visual cortex containing the responses of more than 28,000 neurons across seven mice stimu- lated with thousands of natural images, together with simultaneous behavioral measurements that include running speed, pupil dilation, and eye movements. The benchmark challenge ranked models based on predictive performance for neuronal responses on a held-out test set, and included two tracks for model input limited to either stimulus only (SENSORIUM) or stimulus plus behavior (SENSORIUM+). As a part of the NeurIPS 2022 competition track, we received 172 model submissions from 26 teams, with the winning teams improving our previous state-of-the-art model by more than 15%. Dataset access and infrastructure for evaluation of model predictions will remain online as an ongoing benchmark. We would like to see this as a starting point for regular challenges and data releases, and as a standard tool for measuring progress in large-scale neural system identification models of the mouse visual system and beyond.
@article{willeke2022bsensorium, title= {Retrospective on the SENSORIUM 2022 competition}, author= {Konstantin F. Willeke and Paul G. Fahey and Mohammad Bashiri and Laura Hansel and Max F. Burg and Christoph Blessing and Santiago A. Cadena and Zhiwei Ding and Konstantin-Klemens Lurz and Kayla Ponder and Taliah Muhammad and Saumil S. Patel and Alexander S. Ecker and Andreas S. Tolias and Fabian H. Sinz}, year= {2022}, journal= {PMLR}, }
.bib
K. F. Willeke, P. G. Fahey, M. Bashiri, L. Pede, M. F. Burg, C. Blessing, S. A. Cadena, Z. Ding, K. Lurz, K. Ponder, T. Muhammad, S. S. Patel, A. S. Ecker, A. S. Tolias, F. H. Sinz
The Sensorium competition on predicting large-scale mouse primary visual cortex activity
arXiv preprint arXiv:2206.08666, 2022
show abstract
The neural underpinning of the biological visual system is challenging to study experimentally, in particular as the neuronal activity becomes increasingly nonlinear with respect to visual input. Artificial neural networks (ANNs) can serve a variety of goals for improving our understanding of this complex system, not only serving as predictive digital twins of sensory cortex for novel hypothesis generation in silico, but also incorporating bio-inspired architectural motifs to progressively bridge the gap between biological and machine vision. The mouse has recently emerged as a popular model system to study visual information processing, but no standardized large-scale benchmark to identify state-of-the-art models of the mouse visual system has been established. To fill this gap, we propose the Sensorium benchmark competition. We collected a large-scale dataset from mouse primary visual cortex containing the responses of more than 28,000 neurons across seven mice stimulated with thousands of natural images, together with simultaneous behavioral measurements that include running speed, pupil dilation, and eye movements. The benchmark challenge will rank models based on predictive performance for neuronal responses on a held-out test set, and includes two tracks for model input limited to either stimulus only (Sensorium) or stimulus plus behavior (Sensorium+). We provide a starting kit to lower the barrier for entry, including tutorials, pre-trained baseline models, and APIs with one line commands for data loading and submission. We would like to see this as a starting point for regular challenges and data releases, and as a standard tool for measuring progress in large-scale neural system identification models of the mouse visual system and beyond.
@article{willeke2022sensorium, title= {The Sensorium competition on predicting large-scale mouse primary visual cortex activity}, author= {Konstantin F. Willeke and Paul G. Fahey and Mohammad Bashiri and Laura Pede and Max F. Burg and Christoph Blessing and Santiago A. Cadena and Zhiwei Ding and Konstantin-Klemens Lurz and Kayla Ponder and Taliah Muhammad and Saumil S. Patel and Alexander S. Ecker and Andreas S. Tolias and Fabian H. Sinz}, year= {2022}, journal= {arXiv preprint arXiv:2206.08666}, }
code
.bib

2021

M. A. Weis, K. Chitta, Y. Sharma, W. Brendel, M. Bethge, A. Geiger, A. S. Ecker
Benchmarking Unsupervised Object Representations for Video Sequences
Journal of Machine Learning Research, 2021
@article{Weis2021, title= {Benchmarking Unsupervised Object Representations for Video Sequences}, author= {Marissa A. Weis and Kashyap Chitta and Yash Sharma and Wieland Brendel and Matthias Bethge and Andreas Geiger and Alexander S. Ecker}, year= {2021}, journal= {Journal of Machine Learning Research}, }
code
.bib
K. Lurz, M. Bashiri, K. Willeke, A. Jagadish, E. Wang, E. Y. Walker, S. A. Cadena, T. Muhammad, E. Cobos, A. S. Tolias, A. S. Ecker, F. H. Sinz
Generalization in data-driven models of primary visual cortex
International Conference on Learning Representations, 2021
@inproceedings{lurz2021generalization, title= {Generalization in data-driven models of primary visual cortex}, author= {Konstantin-Klemens Lurz and Mohammad Bashiri and Konstantin Willeke and Akshay Jagadish and Eric Wang and Edgar Y. Walker and Santiago A Cadena and Taliah Muhammad and Erick Cobos and Andreas S. Tolias and Alexander S Ecker and Fabian H. Sinz}, year= {2021}, booktitle= {International Conference on Learning Representations}, }
code
.bib
M. Burg, S. Cadena, G. Denfield, E. Walker, A. Tolias, M. Bethge, A. Ecker
Learning divisive normalization in primary visual cortex
PLOS Computational Biology, 2021
show abstract
Divisive normalization (DN) is a prominent computational building block in the brain that has been proposed as a canonical cortical operation. Numerous experimental studies have verified its importance for capturing nonlinear neural response properties to simple, artificial stimuli, and computational studies suggest that DN is also an important component for processing natural stimuli. However, we lack quantitative models of DN that are directly informed by measurements of spiking responses in the brain and applicable to arbitrary stimuli. Here, we propose a DN model that is applicable to arbitrary input images. We test its ability to predict how neurons in macaque primary visual cortex (V1) respond to natural images, with a focus on nonlinear response properties within the classical receptive field. Our model consists of one layer of subunits followed by learned orientation-specific DN. It outperforms linear-nonlinear and wavelet-based feature representations and makes a significant step towards the performance of state-of-the-art convolutional neural network (CNN) models. Unlike deep CNNs, our compact DN model offers a direct interpretation of the nature of normalization. By inspecting the learned normalization pool of our model, we gained insights into a long-standing question about the tuning properties of DN that update the current textbook description: we found that within the receptive field oriented features were normalized preferentially by features with similar orientation rather than non-specifically as currently assumed.
@article{burg_2021_learning_divisive_normalization, title= {Learning divisive normalization in primary visual cortex}, author= {Burg, Max F. and Cadena, Santiago A. and Denfield, George H. and Walker, Edgar Y. and Tolias, Andreas S. and Bethge, Matthias and Ecker, Alexander S.}, year= {2021}, journal= {PLOS Computational Biology}, }
code
.bib
M. A. Weis, L. Pede, T. Lüddecke, A. S. Ecker
Self-supervised Representation Learning of Neuronal Morphologies
arXiv, 2021
show abstract
Understanding the diversity of cell types and their function in the brain is one of the key challenges in neuroscience. The advent of large-scale datasets has given rise to the need of unbiased and quantitative approaches to cell type classification. We present GraphDINO, a purely data-driven approach to learning a low dimensional representation of the 3D morphology of neurons. GraphDINO is a novel graph representation learning method for spatial graphs utilizing self-supervised learning on transformer models. It smoothly interpolates between attention-based global interaction between nodes and classic graph convolutional processing. We show that this method is able to yield morphological cell type clustering that is comparable to manual feature-based classification and shows a good correspondence to expert-labeled cell types in two different species and cortical areas. Our method is applicable beyond neuroscience in settings where samples in a dataset are graphs and graph-level embeddings are desired.
@article{weis2021selfsupervised, title= {Self-supervised Representation Learning of Neuronal Morphologies}, author= {Marissa A. Weis and Laura Pede and Timo Lüddecke and Alexander S. Ecker}, year= {2021}, journal= {arXiv}, }
code
.bib
D. Kobak, Y. Bernaerts, M. Weis, F. Scala, A. Tolias, P. Berens
Sparse reduced-rank regression for exploratory visualisation of paired multivariate data
Journal of the Royal Statistical Society: Series C (Applied Statistics), 2021
@article{weis2021sparse, title= {Sparse reduced-rank regression for exploratory visualisation of paired multivariate data}, author= {Kobak, Dmitry and Bernaerts, Yves and Weis, Marissa A. and Scala, Federico and Tolias, Andreas S. and Berens, Philipp}, year= {2021}, journal= {Journal of the Royal Statistical Society: Series C (Applied Statistics)}, }
code
.bib

2020

V. Benson, A. Ecker
Assessing out-of-domain generalization for robust building damage detection
NeurIPS 2020 Workshop on Artificial Intelligence for Humanitarian Assistance and Disaster Response (AI+HADR 2020), 2020
@inproceedings{benson2020assessing, title= {Assessing out-of-domain generalization for robust building damage detection}, author= {Benson, Vitus and Ecker, Alexander}, year= {2020}, booktitle= {NeurIPS 2020 Workshop on Artificial Intelligence for Humanitarian Assistance and Disaster Response (AI+HADR 2020)}, }
code
.bib
C. Michaelis, M. Bethge, A. Ecker
Closing the generalization gap in one-shot object detection
arXiv preprint arXiv:2011.04267, 2020
@article{michaelis2020closing, title= {Closing the generalization gap in one-shot object detection}, author= {Michaelis, Claudio and Bethge, Matthias and Ecker, Alexander S}, year= {2020}, journal= {arXiv preprint arXiv:2011.04267}, }
.bib
T. Lüddecke, A. Ecker
CNNs efficiently learn long-range dependencies
NeurIPS 2020 Workshop on Shared Visual Representations in Human & Machine Intelligence, 2020
@inproceedings{luddeckecnns, title= {CNNs efficiently learn long-range dependencies}, author= {Lüddecke, Timo and Ecker, Alexander}, year= {2020}, booktitle= {NeurIPS 2020 Workshop on Shared Visual Representations in Human & Machine Intelligence}, }
.bib
T. Lüddecke, F. Wörgötter
Fine-grained action plausibility rating
Robotics and Autonomous Systems (RAS), 2020
@article{lueddecke20, title= {Fine-grained action plausibility rating}, author= {Lüddecke, Timo and Wörgötter, Florentin}, year= {2020}, journal= {Robotics and Autonomous Systems (RAS)}, }
.bib
M. Rolínek, V. Musil, A. Paulus, M. Vlastelica, C. Michaelis, G. Martius
Optimizing Rank-based Metrics with Blackbox Differentiation
Computer Vision and Pattern Recognition (CVPR), 2020
@inproceedings{Rolínek2019a, title= {Optimizing Rank-based Metrics with Blackbox Differentiation}, author= {M. Rolínek and V. Musil and A. Paulus and M. Vlastelica and C. Michaelis and G. Martius}, year= {2020}, booktitle= {Computer Vision and Pattern Recognition (CVPR)}, }
.bib
I. Ustyuzhaninov, S. A. Cadena, E. Froudarakis, P. G. Fahey, E. Y. Walker, E. Cobos, J. Reimer, F. H. Sinz, A. S. Tolias, M. Bethge, A. S. Ecker
Rotation-invariant clustering of functional cell types in primary visual cortex
International Conference on Learning Representations (ICLR), 2020
@inproceedings{Ustyuzhaninov2020a, title= {Rotation-invariant clustering of functional cell types in primary visual cortex}, author= {I. Ustyuzhaninov and S. A. Cadena and E. Froudarakis and P. G. Fahey and E. Y. Walker and E. Cobos and J. Reimer and F. H. Sinz and A. S. Tolias and M. Bethge and A. S. Ecker}, year= {2020}, booktitle= {International Conference on Learning Representations (ICLR)}, }
.bib
Z. Zhao, D. Klindt, A. M. Chagas, K. P. Szatko, L. Rogerson, D. Protti, C. Behrens, D. Dalkara, T. Schubert, M. Bethge, K. Franke, P. Berens, A. S. Ecker, T. Euler
The temporal structure of the inner retina at a single glance
Scientific Reports, 2020
@article{Z*2020a, title= {The temporal structure of the inner retina at a single glance}, author= {Z. Zhao and D. Klindt and A. M. Chagas and K. P. Szatko and L. Rogerson and D. Protti and C. Behrens and D. Dalkara and T. Schubert and M. Bethge and K. Franke and P. Berens and A. S. Ecker and T. Euler}, year= {2020}, journal= {Scientific Reports}, }
.bib

2019

A. S. Ecker, F. H. Sinz, E. Froudarakis, P. G. Fahey, S. A. Cadena, E. Y. Walker, E. Cobos, J. Reimer, A. S. Tolias, M. Bethge
A rotation-equivariant convolutional neural network model of primary visual cortex
International Conference on Learning Representations (ICLR), 2019
@inproceedings{ecker_2019, title= {A rotation-equivariant convolutional neural network model of primary visual cortex}, author= {A. S. Ecker and F. H. Sinz and E. Froudarakis and P. G. Fahey and S. A. Cadena and E. Y. Walker and E. Cobos and J. Reimer and A. S. Tolias and M. Bethge}, year= {2019}, journal= {International Conference on Learning Representations (ICLR)}, }
code
.bib
C. Michaelis, B. Mitzkus, R. Geirhos, E. Rusak, O. Bringmann, A. S. Ecker, M. Bethge, W. Brendel
Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming
Machine Learning for Autonomous Driving Workshop, NeurIPS 2019, 2019
@inproceedings{michaelis2019dragon, title= {Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming}, author= {C. Michaelis and B. Mitzkus and R. Geirhos and E. Rusak and O. Bringmann and A. S. Ecker and M. Bethge and W. Brendel}, year= {2019}, booktitle= {Machine Learning for Autonomous Driving Workshop, NeurIPS 2019}, }
.bib
T. Lüddecke, T. Kulvicius, F. Wörgötter
Context-based Affordance Segmentation from 2D Images for Robot Action
Robotics and Autonomous Systems (RAS), 2019
@article{lueddecke19a, title= {Context-based Affordance Segmentation from 2D Images for Robot Action}, author= {Timo Lüddecke and Tomas Kulvicius and Florentin Wörgötter}, year= {2019}, journal= {Robotics and Autonomous Systems (RAS)}, }
.bib
S. A. Cadena, G. H. Denfield, E. Y. Walker, L. A. Gatys, A. S. Tolias, M. Bethge, A. S. Ecker
Deep convolutional models improve predictions of macaque V1 responses to natural images
PLoS Computational Biology, 2019
@article{Cadena2019, title= {Deep convolutional models improve predictions of macaque V1 responses to natural images}, author= {S. A. Cadena and G. H. Denfield and E. Y. Walker and L. A. Gatys and A. S. Tolias and M. Bethge and A. S. Ecker}, year= {2019}, journal= {PLoS Computational Biology}, }
.bib
T. Lüddecke, A. Agostini, M. Fauth, M. Tamosiunaite, F. Wörgötter
Distributional Semantics of Objects in Visual Scenes in Comparison to Text
Artificial Intelligence, 2019
@article{lueddecke19, title= {Distributional Semantics of Objects in Visual Scenes in Comparison to Text}, author= {Timo Lüddecke and Alejandro Agostini and Michael Fauth and Minija Tamosiunaite and Florentin Wörgötter}, year= {2019}, journal= {Artificial Intelligence}, }
.bib
S. A. Cadena, F. H. Sinz, T. Muhammad, E. Froudarakis, E. Cobos, E. Y. Walker, J. Reimer, M. Bethge, A. Tolias, A. S. Ecker
How well do deep neural networks trained on object recognition characterize the mouse visual system?
NeurIPS Neuro AI Workshop, 2019
@inproceedings{Cadena2019b, title= {How well do deep neural networks trained on object recognition characterize the mouse visual system?}, author= {S. A. Cadena and F. H. Sinz and T. Muhammad and E. Froudarakis and E. Cobos and E. Y. Walker and J. Reimer and M. Bethge and A. Tolias and A. S. Ecker}, year= {2019}, journal= {NeurIPS Neuro AI Workshop}, }
.bib
R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, W. Brendel
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
International Conference on Learning Representations (ICLR), 2019
@inproceedings{Geirhos2019a, title= {ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness}, author= {R. Geirhos and P. Rubisch and C. Michaelis and M. Bethge and F. A. Wichmann and W. Brendel}, year= {2019}, journal= {International Conference on Learning Representations (ICLR)}, }
.bib
E. Y. Walker, F. H. Sinz, E. Froudarakis, P. G. Fahey, T. Muhammad, A. S. Ecker, E. Cobos, J. Reimer, X. Pitkow, A. S. Tolias
Inception loops discover what excites neurons most using deep predictive models
Nature Neuroscience, 2019
@article{Walker2019, title= {Inception loops discover what excites neurons most using deep predictive models}, author= {E. Y. Walker and F. H. Sinz and E. Froudarakis and P. G. Fahey and T. Muhammad and A. S. Ecker and E. Cobos and J. Reimer and X. Pitkow and A. S. Tolias}, year= {2019}, journal= {Nature Neuroscience}, }
.bib
Neural Data Science Group
Institute of Computer Science
University of Goettingen