3D L2S-Microscopy for Unstained Cell Clusters

• Applicant: Professor Dr. Ivo Ihrke
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Electronic Semiconductors, Components and Circuits, Integrated Systems, Sensor Technology, Theoretical Electrical Engineering
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 459284860

The subproject investigates time-resolved 3D microscopy as an application of the L2S paradigm, creating one of the proposed adaptive L2S sensor systems in the process.It aims at developing an experimental coded microscopy platform for 3D and time-varying 3D (=4D) microscopy of unstained cell clusters, the imaging of which is of high importance in biomedical research. We aim at creating a flexible research tool rather than a targeted microscope solution. It is envisaged that the microscopy hardware platform enables different physical coding strategies that can be completely configured by software. This, in turn, enables the application of L2S techniques for its application-specific optimization. The subproject develops all necessary hardware and software aspects for this application. For the machine learning aspects, i.e. network architecture, loss function design and training schemes a tight collaboration with the respective machine learning projects is envisaged. The fundamental research question targeted in this subproject is: Is it possible to unlock new performance regions and/or significantly expanded conditions of applicability (thick samples, combined diffraction and absorption) in time-varying 3D microscopy using specifically developed AI techniques in conjunction with hardware/software codesign?To this end, the subproject investigates 4 main topics: 1. Training Data and Ground Truth Generation, Data Acquisition, 2. Modeling, System-Simulation and Differentiable Digital Twin, 3. Microscope Construction, Calibration, and Validation, and 4. Learning to Sense techniques for 3D Refractive Index and Absorption Microscopy.Topic 2 benefits from joint activities with subproject P5 (Andreas Kolb). Topic 4 will be realized in close collaboration with subprojects P1 (Michael Möller), P2 (Margret Keuper) and P3 (Volker Blanz).

• DFG Programme: Research Units
• Subproject of: FOR 5336: KI-FOR Learning to Sense
• International Connection: France
• Cooperation Partners: Professor Dr. Olivier Haeberlé; Professor Pierre Nassoy, Ph.D.