Learning Concepts in Deep Networks

• Applicant: Professor Dr. Klaus-Robert Müller
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Software Engineering and Programming Languages
• Term: from 2012 to 2017
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 227351812

Learning appropriate representations, or extracting useful featuresfrom data, is one of the fundamental problems of MachineLearning. Recently a number of methods have been developed forlearning deep representations (e.g.\ deep neural networks or deepprobabilistic graphical models). While existing research hasempirically validated the benefit of deep learning, there is currentlya lack in deeper understanding of deep architectures and theirrepresentations. Open questions are: Are deep representationsfundamentally different to kernels, or can they not be understood as aspecial type of kernel? What are characteristics of deeprepresentations that make them beneficial?This project is organized in two parts. In the first (analytical) partwe will develop generative and discriminative methods toanalyze learning concepts in deep networks. We anticipate that thisanalysis will allow for a unified view on kernels and representations,overcoming the false dichotomy between so-called deep and shallowrepresentations. In the second (constructive) part, we will utilizethese analytical tools for developing alternative methods to learndeep representations. Our approach will be to select good deeprepresentations from massive sets of randomized proposal structuresusing the analytical measures to be developed initially.The outcome of this project will be (1) methods to precisely quantifythe characteristics and benefits of deep learning and (2)concepts for constructing improved deep learning based on thesemeasures.

• DFG Programme: Research Grants
• Participating Person: Professor Dr. Marc Toussaint