Deep Learning for Imaging Non-Static Objects

• Applicant: Professor Dr. Reinhard Heckel
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Communication Technology and Networks, High-Frequency Technology and Photonic Systems, Signal Processing and Machine Learning for Information Technology
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 517586365

Deep neural networks have emerged as highly successful tools for image reconstruction problems. For example, neural networks significantly accelerate magnetic resonance imaging (MRI), an important medical imaging technique. This success is largely attributed to neural networks being able to learn excellent priors (models) for static objects from data. However, objects in many important scientific and medical imaging systems change significantly during measurement acquisition. For example, while acquiring cardiac MRI data, the heart goes through hundreds of cardiac cycles. Second, a protein imaged with cryogenic electron microscopy (cryo-EM) is in a continuum of configurations during acquisition. Current methods for imaging such non-static objects cast reconstruction as a series of static reconstruction problems. This induces a systematic error and significantly limits image quality. The goal of this project is to develop new deep learning techniques for imaging non-static objects. By modeling non-static objects with neural networks, we will address two outstanding problems in imaging: enabling free-breathing MRI and inferring high-resolution continuous 3D-configurations of proteins with cryo-EM. First, we will develop methods that cast imaging of a non-static object as fitting a neural network to measurement data. This approach will, for the first time, enable high-resolution free-breathing cardiac MRI, and more broadly alleviate motion artifacts in MRI. Second, we will develop networks trained end-to-end to reconstruct non-static objects, which will enable fast high-resolution cine cardiac MRI. Third, we will develop methods imposing strong learned neural network priors on proteins for enabling inferring the continuous 3D-configurations of proteins with cryo-EM, considered the holy grail of structural biology. Finally, we will develop mathematical reconstruction and robustness guarantees for imaging non-static objects.

• DFG Programme: Research Grants