Multi-Modal Image Registration of 3D Ultrasound and Magnet Resonance Tomography Data for Applications in Neurosurgery and Prostate Cancer Treatment

• Applicant: Dr. Tassilo Klein
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2013 to 2015
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 241307562

Nowadays medical treatment is largely unthinkable without the support of the various imaging technologies available. Typically a multitude of images from different modalities are required for specific diagnosis and during various stages of interventions. The choice of image modalities available today is quite wide and further extended by specific acquisition protocols. Among the most common modalities are magnet resonance (MR), computed tomography (CT), ultrasound (US) and positron emission tomography (PET). As a result of these different images, physicians like to have them transformed so that important structures overlap each other correctly. The process of finding the appropriate transformation in order to achieve this desired overlap is commonly referred to as registration. Image registration and in particular multi-modal image registration has become an increasingly important and an actively researched domain. One of the main reasons for the importance is the multi-modal nature of interventions. Pre-operatively, images are generated for planning the intervention. However, planning data and corresponding image data are subject to a progressive degradation in accuracy during an intervention due factors such as organ deformation. Therefore intra-operative images are required that capture the current position and condition of organs. In this respect, it is not uncommon for intra-operative and pre-operative images to be quite different in terms of modality, quality, viewpoint, or other parameters. As an example, in time critical events, such as emergencies the image modality available is required to be accessible without logistic effort and provide real-time information. Therefore, US is often chosen for intra-operative imaging, whereas for planning MR and CT is typically preferred due to repeatability and reliability. Finally, given pre-operative and intra-operative data, mutual registration provides valuable information in terms of deformation that can be used to update pre-operative planning data.In the context of research project, methods shall be investigated on how to register 3D US data with corresponding MR image data, a problem that is still generally considered unsolved. Registration of these modalities has largely remained a challenge due to the lack of clear features in US images and the different physical properties underlying these modalities. In order to address the problem of registration, methods are investigated on how to take advantage of US radio frequency (RF). RF data is the signal from which in the processing pipeline of US machines the commonly known B-mode image is generated. The comparably high resolution and the raw signal characteristics make RF data particularly attractive for image processing. Primary applications for the proposed multi-modal registration are neurosurgical interventions such as biopsy and tumor resection. Secondary applications are of interest are similar interventions on the prostate.

• DFG Programme: Research Fellowships
• International Connection: USA
• Host: Professor William M. Wells III