Towards a Statistical Analysis of DNN Training Trajectories

• Applicant: Professor Dr. Ingo Steinwart
• Subject Area: Mathematics
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 464110610

Deep neural networks (DNNs) have become one of the state-of-the-art machine learning methods in many areas of applications. Despite this success and the fact that these methods have been considered for around 40 years, our current statistical understanding of their learning mechanisms is still rather limited. Part of the reasons for this lack of understanding is the fact that in many cases the tools of classical statistical learning theory can no longer be applied. The overall goal of this project is to establish key aspects of a statistical analysis of DNN training algorithms that are close to the ones used in practice. In particular, we will investigate the statistical properties of trajectories produced by (variants of) gradient descent, where the focus lies on the question whether such trajectories contain predictors with good generalization guarantees.

• DFG Programme: Priority Programmes
• Subproject of: SPP 2298: Theoretical Foundations of Deep Learning