Combinatorial and implicit approaches to deep learning - Phase II

• Applicant: Professor Dr. Guido Montúfar
• Subject Area: Mathematics ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 464109215

This project develops theoretical foundations of deep learning focusing on combinatorial and algebraic geometric structures in learning with neural networks. We build bridges to algebraic statistics and tropical geometry in order to advance the theory at three different levels: individual functions, function classes, and function classes together with a training objective. We exploit techniques on combinatorics of arrangements, algebraic implicitization, and constrained optimization, to give a tight description of finite networks, with explicit control on the role of the data. A key innovation that we expand are implicit descriptions of the function classes represented by neural networks. We turn our attention to new challenges in the optimization of mildly overparametrized networks, exploration of activation functions, networks subject to parameter constraints, and the development of systematic approaches to characterize training invariants.

• DFG Programme: Priority Programmes
• Subproject of: SPP 2298: Theoretical Foundations of Deep Learning