Enriching Deep Learning with Probability and Geometry

• Applicant: Professor Dr. Philipp Hennig
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Theoretical Computer Science
• Term: since 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 543917411

Laplace approximations have re-emerged as a potent and efficient tool for deep learning. They combine the two powerful paradigms of automatic differentiation and numerical linear algebra to enable functionality that had previously become niche due its high computational cost. In particular, Laplace approximations yield a Bayesian formalism for deep learning, effectively turning any deep neural network into an approximate Gaussian process. But they also define a metric, and an associated manifold to the deep network and its parameter space. This proposal to the SPP hopes to expand recent results both in a theoretical and algorithmic direction. On the theoretical side, the project aims to leverage differential geometry to improve understanding of the computational complexity of Bayesian deep training. As a direct outcome, the project will then develop new algorithms and functional extensions of deep learning through re-parametrization, to provide better calibrated uncertainty quantification in deep learning.

• DFG Programme: Priority Programmes
• Subproject of: SPP 2298: Theoretical Foundations of Deep Learning