AUTHOR=Wei Zhihao , Liu Xi , Yan Ruiqing , Sun Guocheng , Yu Weiyong , Liu Qiang , Guo Qianjin 

TITLE=Pixel-level multimodal fusion deep networks for predicting subcellular organelle localization from label-free live-cell imaging

JOURNAL=Frontiers in Genetics

VOLUME=Volume 13 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/genetics/articles/10.3389/fgene.2022.1002327

DOI=10.3389/fgene.2022.1002327

ISSN=1664-8021

ABSTRACT=Complex intracellular organization are common represented by dividing the metabolic process of cells into different organelles. Therefore, identifying sub-cellular organelle architecture is significant for understanding intracellular structural properties, specific functions and biological processes in cells. However, the discrimination of these structures in the natural organizational environment and their functional consequences are not clear. In this paper, we propose a new Pixel Level Multimodal Fusion (PLMF) Deep Networks which can be used to predict the location of cellular organelle using label-free cell optical microscopy images followed by deep-learning-based automated image denoising. It provides valuable insights that can be of tremendous help to improve the specificity of label-free cell optical microscopy by using the Transformer-Unet network to predict the ground truth imaging which are correspond to different sub-cellular organelle architecture. The new prediction method proposed in this paper combines the advantages of transformer's global prediction and CNN's local detail analysis ability of background features for label-free cell optical microscopy images , so as to improve the prediction accuracy. Our experimental results showed that the PLMFnetwork can achieve over 0.91 Pearson’s correlation coefficient (PCC) correlation between estimated and true fractions on lung cancer cells imaging datasets. In addition, we applied the PLMFnetwork method on the cell images for label-free predicting several different subcellular components simultaneously, rather than using several fluorescent labels. These results open up a new way for the time-resolved study of subcellular components in different cells, especially for cancer cells.