Robust Nucleus Detection with Deep Learning for Automation of Industry Relevant Assays
SBI2 High Content, Virtual Meeting
September 17, 2020
Detection of nuclei is a standard analysis procedure, and usually forms the initial step in image analysis. Its detection performance degrades in more heterogeneous images, e.g. if cells appear with very different light intensities, or if cells clump together.
The 2018 Data Science Bowl winner solutions for nucleus segmentation overcame this limitation. One team used the Mask-R-CNN model. We have adapted Mask-R-CNN, using a refined network training schema combining a tuned parameter set and pre-training and determined its robustness and efficiency on, industry-relevant data sets. Comparing the results to those obtained by Cell Profiler, we observed an enhanced performance of the adapted Mask-R-CNN model (robustly identify small nuclei in dense low magnification images, and segmented cells also in the presence of clumping). It thus overcomes several of the cellular heterogeneity issues and as a plus can be further adapted to new, hitherto unknown situations.