Pathology Segmentation Learned from Weakly Annotated Medical Images

As written in the coverletter, the proposed work is an extension of the recently granted CTI project 27395.1 on brain shift correction for Neurosurgical interventions. The aim of the CTI project is to develop techniques to determine the brain shift and then to overlay the tumour and other critical structures onto the surgical microscope's image. This, however, implies that segmentation of the tumour, brain surface, vascular tree and the critical structures are available. Segmenting these often requires substantial manual input for training which is a tedious and time consuming task. With the research described herein (partially funded by a Novartis FreeNovation Project) we try to close this gap and go one step beyond. In particular we propose an approach able to learn on its own how to segment a pathology only on weakly labelled data. In other words our approach is capable to learn how to segment pathologies from a training set of images with the pathology and a second set of images without the pathology (i.e. healthy subjects). Such data sets are easy to get in contrast to the manually labelled data sets required for the state-of-the-art approaches. The proposed approaches is the first of its kind and inspired by CycleGAN (a DeepLearning domain transfer approach) [1]. Our approach can model pathologies in medical data trained only with data labelled on the image level (i.e. healthy vs. diseased). Not only can the model create pixelwise semantic segmentations of the pathologies it can also create inpaintings (i.e. heal) to render the pathological image healthy. As a side effect, we can also create new unseen pathological samples useful for example in training of medical personnel. In a proof-of-principle study we could recently show that the idea has great potential and might even be a disruptive technology in image segmentation. The significance of the proposed project is very high as it might render manual segmentation unnecessary in the near future. Training the algorithm to recognise and segment new pathologies would be simple and fast. Imaging CROs could evaluate their drug studies more cost effective and also faster speeding up the development cycle of new drugs. In this research proposal we will first give an overview on the principles and limitations of current in segmentation concepts, followed by an overview of our own research directions in this field and the detailed research plan. After the project plan and risk analysis the significance of the planned work is shown. Lastly, the budget for the planned research is detailed.