Animal Behavior Recognition in the Wild

Structure of behavior analysis tasks: Starting from a video taken in the wild, the goal is to obtain a structured representation of actions and interactions between the depicted animals while keeping track of their identities. The behavior analysis output on the right shows the behavior of two individuals (blue and violet) over time.

Motivation

Animal behavior recognition is central to research in cognition, evolution, and conservation. In lab settings, computer vision tools for animal behavior recognition are already well established and considerably accelerate research. In more complex in-the-wild settings however, vision models do not yet perform well enough for widespread adaptation. Our goal is to advance animal behavior recognition, especially for primates. The tasks we are trying to solve are mainly spatio-temporal action recognition, but we are also looking at individual re-identification, tracking and detection. For primate behavior, we have large amounts of unlabeled data available, but labels are scarce and expensive. We are thus especially interested in approaches that utilize self-supervised learning, weakly-supervised learning or transfer learning.

Project

This is a generic opening. If you are interested in working on animal behavior recognition, contact me to discuss a specific Master thesis scope. Potential areas for a thesis are

• self-supervised pretraining on large-scale unlabeled data to improve primate behavior recognition
• test-time training or domain adaptation for transfer learning between behavior datasets
• reducing background bias in behavior recognition
• animal re-identification
• behavior recognition on 2D or 3D animal poses
• zero-shot or few-shot behavior recognition using vision foundation models
• animal pose estimation or reconstruction from monocular video

Prerequisites

The following skills are required:

• Familiar with Python (Pytorch, Numpy, Scipy would be helpful)
• Basic machine learning and deep learning knowledge

Contact

To apply please email Felix Müller stating your interest in this project and detailing your relevant skills. A part of this project could be also a lab rotation.