Free probability aspects of neural networks

• Applicant: Professor Dr. Roland Speicher
• Subject Area: Mathematics
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 461815964

Machine learning and, in particular, deep neural networks have made tremendous practical progress in the last couple of years, with many successful applications over a wide range of disciplines. However, the mathematical foundation of this success is still not well understood.A neural network, in its mathematical incarnation, is a quite general composition of matrices and entry-wise applied non-linear functions. In the limit of infinite width the matrix part is amenable to methods from random matrix and free probability theory. The inclusion of non-linearities in random matrix considerations is a quite recent challenge, mainly motivated by the relevance for neural networks. In the last few years, there have been approaches relying on random matrix and/or free probability theory to investigate questions around deep learning. The goal of the present project is to streamline and generalize those previous investigations and put them on a more systematic foundation.

• DFG Programme: Research Grants