Generative Deep Learning for Lidar Scans of Trees

Generative models have been very successful in recent years to generate images and text. More recently, generative models for 3d structures in the form of meshes or point clouds have shown decent results [1]. In this thesis, we want to build generative models for lidar scans of trees. Trees are highly intricate structures, consisting of many small branches and lots of details, which makes the prediction task more challenging. Recently, several large scale datasets of single tree lidar scans or segmented lidar point clouds have been published [2, 3]. These can be used as training data.

Method

We intend to build on top of denoising-diffusion methods and flow-matching methods [4], that aim to learn an iterative generation process from random noise to a data point. A particular question for tree point cloud generation is how to scale the methods to the high amount of points typically encountered in tree point clouds.

Details

For this thesis, basic familiarity with deep learning and pytorch is a prerequisite. The first step will be constructing a codebase based on the chosen generative modeling framework and overfitting it to generate a single tree. The second step would be to take the union of all trees and use generative modeling to generate an individual tree. The tree would be the equivalent of the class for class-conditional guidance. Step 3 would be to guide by an embedding vector of each tree.

Contact

To apply please email Jan van Delden stating your interest in this project and detailing your relevant skills. Please note that this project is better suited for a student who already has a basic familiarity with deep learning and pytorch.

References

• [1] “Coordinate In and Value Out: Training Flow Transformers in Ambient Space”. In: Submitted to The Thirteenth International Conference on Learning Representations. under review. 2024. url: https://openreview.net/forum?id=AHnj6YbNbj.
• [2] Stefano Puliti et al. “Benchmarking tree species classification from proximally-sensed laser scanning data: introducing the FOR-species20K dataset”. In: arXiv preprint arXiv:2408.06507 (2024)
• [3] Jonathan Henrich, Jan van Delden, Dominik Seidel, Thomas Kneib, and Alexander Ecker. “Tree-Learn: A deep learning method for segmenting individual trees from ground-based LiDAR forest point clouds”. In: Ecological Informatics (2024), p. 102888.
• [4] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. “Flow matching for generative modeling”. In: arXiv preprint arXiv:2210.02747 (2022).