• Homepage
  • mmc2021 Abstract Database
  • A comparison of convolutional neural network-based approaches for label-free cell cycle prediction
  • A comparison of convolutional neural network-based approaches for label-free cell cycle prediction

    Abstract number
    Presentation Form
    Corresponding Email
    [email protected]
    Poster Session 1
    Elsa Sörman Paulsson (1), Rickard Sjögren (1)
    1. Sartorius Corporate Research

    Convolutional Neural Networks

    Phase Contrast Microscopy

    Label-free Image Analysis

    Cell Cycle Detection

    Deep Learning

    Abstract text

    A fundamental aspect of cell biology research is to interrogate cell-cycle dynamics, requiring accurate identification of a given cell’s position within the cell cycle. In microscopic imaging it is standard practice to use fluorescent probes, such as the Fluorescent Ubiquitination-based Cell Cycle Indicator (FUCCI), to create a strong signal of the cell cycle state. Although fluorescent imaging provides a strong signal to help analysis and has been used in countless biological discoveries, there is mounting evidence that fluorescent sensors can alter biological responses stressing the need for label-free approaches. While there are sophisticated label-free imaging technologies facilitating analysis, simple brightfield and phase contrast imaging remains widespread due to being cheap, accessible, and easy-to-use. Thanks to great advances in deep learning-based image analysis driven by convolutional neural networks (CNNs), image analysis-workflows are now more capable than ever before and there are many promising ways of how to determine cell cycle state directly from label-free images.

    We compare two different approaches to use CNN-based machine learning to determine cell cycle state directly from label-free 2D phase contrast microscopy-images. Both approaches use fluorescent images to set up ground truth for machine learning-algorithms to learn to predict the corresponding fluorescent readout for future label-free images. One approach is based on first segmenting single cells and then assigning them into discrete categories based on the fluorescent signal and finally train a CNN-classifier to predict the category for future cells. The second approach uses an in-silico labelling (ISL) approach (Christiansen 2018) and train a CNN to predict the corresponding fluorescent image directly from the phase contrast images and thereafter segmenting them and assigning them into categories based on the CNN-predicted fluorescence (Rappez 2020).

    In a case study on cell cycle markers, we used a dataset of SK-OV-3, THP-1 and MDA-MB-231 cells labelled with two-color FUCCI. To perform single cell classification, we first performed label-free cell segmentation using a CNN-based instance segmentation model trained on LIVECell, our recently developed cell segmentation dataset (Edlund 2021). We then fine-tuned a CNN-classifier based on ResNet50 and pretrained on ImageNet to classify individual cells assigned to four classes according to their FUCCI expression. In parallel, we trained an ISL CNN, a modified variant of U-net (Ronneberger 2015) that we call OSA-U-net using one-shot aggregation (OSA) and effective Squeeze-and-Excitation blocks (Lee 2020), to minimize the difference between predicted and measured FUCCI fluorescent images as weighted by a smooth L1-loss. We then segmented the cells and assigned them into classes based on the ISL expression. We found that the ISL approach achieved better classification performance compared the classification approach, F1-score of 83.3 and 64 % for the two channels respectively compared to 79.2 and 53.6 % of the classifier. 

    The ISL approach not only performed better at cell cycle classification but it also provides less involved configuration compared to the segment-than-classify approach. The classifier performance is directly dependent on the segmentation accuracy as well as the segmentation post-processing. In comparison, the ISL approach only requires phase contrast and fluorescent image pairs and cell segmentation is a completely separate step making it easier to use. To conclude, our ISL-based workflow provides an easy-to-configure method with promising performance for label-free cell cycle detection.


    Christiansen, E. M., Yang, S. J., Ando, D. M., Javaherian, A., Skibinski, G., Lipnick, S., ... & Finkbeiner, S. (2018). In silico labeling: predicting fluorescent labels in unlabeled images. Cell, 173(3), 792-803.

    Edlund, C., et al. “LIVECell - A large-scale dataset for label-free live cell segmentation” Nature Methods (in review) (2021)

    Rappez, L., Rakhlin, A., Rigopoulos, A., Nikolenko, S., & Alexandrov, T. (2020). DeepCycle reconstructs a cyclic cell cycle trajectory from unsegmented cell images using convolutional neural networks. Molecular systems biology, 16(10), e9474.

    Ronneberger, O., Fischer, P., & Brox, T. (2015, October). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234-241). Springer, Cham.

    Lee, Y., & Park, J. (2020). Centermask: Real-time anchor-free instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 13906-13915).