Minimal supervision for phase-contrast microscopy segmentation

• Applicant: Professor Dr. Constantin Pape
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Cell Biology
• Term: since 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 530597088

Microscopy is widely used to study the processes of life and modern microscopes produce large datasets. For example, phase-contrast microscopy is a non photo-toxic technique that is used in live-cell imaging to observe the development of cell cultures or tissues over time. Such experiments are routinely conducted in a high-throughput fashion, resulting in many images per time point. For quantitative analysis of these large datasets the time consuming step of identifying and delineating the cells has to be automated. This task can be formalized as a segmentation problem, a well studied computer vision task. Over the last decade deep learning methods have been adopted for many computer vision tasks, including cell segmentation and yield results that are often sufficiently good for direct further analysis. However, this high quality comes at the price of large amounts of (manually) labeled training data, which must be very similar to the data at hand. Creating such labels can only be done by experts and is very time consuming. In practice this requirement often makes the use of state-of-the-art methods infeasible and prevents quantitative analysis of large datasets. To overcome this limitation I plan to develop segmentation methods that require minimal supervision: methods that can be trained with only a fraction of the labels necessary for current methods while producing results of similar quality. They will be based on recent advances in self-supervised learning, which formulates a pretext task without labels to learn a good data representation, and domain adaptation. The latter adapts a model trained on a source dataset with labels to the target dataset in a manner that aims to preserve the segmentation quality. Both techniques have been successfully applied in natural images, but their extension to microscopy segmentation has so far proven difficult. The proposed project contains four specific objectives: i) Build a large dataset of phase-contrast microscopy images (without labels) in collaboration with a live-cell imaging facility. ii) Develop self-supervised learning methods for phase-contrast microscopy. These methods require a large amount of (unlabeled) data and will be developed using the dataset from i). iii) Develop domain adaptation methods for phase-contrast microscopy. iv) Make the methods from ii) and iii) available to a wide range of users by providing them as plugins for a common image analysis platform. The dataset from i) and methods developed in ii) will also shed light on the applicability of self-supervised learning beyond natural images, a research question that is currently posed due to the surprising success of these methods for large-scale natural image datasets. While this project focuses on phase-contrast microscopy, the proposed methods will provide a starting point for developing similar techniques for other microscopy modalities like fluorescence or electron microscopy.

• DFG Programme: Research Grants
• Co-Investigator: Professor Dr. Argyris Papantonis