2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于逐像素点深度卷积网络分割模型的上皮和间质组织分割

骆小飞 徐军 陈佳梅

骆小飞, 徐军, 陈佳梅. 基于逐像素点深度卷积网络分割模型的上皮和间质组织分割. 自动化学报, 2017, 43(11): 2003-2013. doi: 10.16383/j.aas.2017.c160464
引用本文: 骆小飞, 徐军, 陈佳梅. 基于逐像素点深度卷积网络分割模型的上皮和间质组织分割. 自动化学报, 2017, 43(11): 2003-2013. doi: 10.16383/j.aas.2017.c160464
LUO Xiao-Fei, XU Jun, CHEN Jia-Mei. A Deep Convolutional Network for Pixel-wise Segmentation on Epithelial and Stromal Tissues in Histologic Images. ACTA AUTOMATICA SINICA, 2017, 43(11): 2003-2013. doi: 10.16383/j.aas.2017.c160464
Citation: LUO Xiao-Fei, XU Jun, CHEN Jia-Mei. A Deep Convolutional Network for Pixel-wise Segmentation on Epithelial and Stromal Tissues in Histologic Images. ACTA AUTOMATICA SINICA, 2017, 43(11): 2003-2013. doi: 10.16383/j.aas.2017.c160464

基于逐像素点深度卷积网络分割模型的上皮和间质组织分割

doi: 10.16383/j.aas.2017.c160464
基金项目: 

江苏创新创业团队人才计划 JS201526

国家自然科学基金 61273259

江苏省自然科学基金 BK20141482

国家自然科学基金 61771249

江苏省"六大人才高峰"高层次人才项目资助计划 2013-XXRJ-019

详细信息
    作者简介:

    骆小飞 2013年和2016年在南京信息工程大学获得学士和硕士学位.主要研究方向为机器学习, 计算机视觉, 医学图像分析.E-mail:luoxiaofeifly@163.com

    陈佳梅 武汉大学中南医院博士研究生.2012年获得武汉大学学士学位.主要研究方向为基于计算机图像和分子探针影像分析的乳腺癌生物学行为研究.E-mail:cjm7352@163.com

    通讯作者:

    徐军 南京信息工程大学教授.主要研究方向为计算病理学, 数字病理, 癌症的计算机辅助检测、诊断与预后, 基于深度学习及大数据驱动的医学数据分析, 临床转化医学.本文通信作者.E-mail:jxu@nuist.edu.cn

A Deep Convolutional Network for Pixel-wise Segmentation on Epithelial and Stromal Tissues in Histologic Images

Funds: 

Jiangsu Innovation and Entrepreneurship Group Talents Plan JS201526

National Natural Science Foundation of China 61273259

Natural Science Foundation of Jiangsu Province BK20141482

National Natural Science Foundation of China 61771249

Six Major Talents Summit of Jiangsu Province 2013-XXRJ-019

More Information
    Author Bio:

    Received his bachelor and master degrees at Nanjing University of Information Science and Technology in 2013 and 2016, respectively. His research interest covers machine learning, computer vision, and medical image analysis

    Ph. D. candidate at Zhongnan Hospital, Wuhan University. She received her bachelor degree at Wuhan University in 2012. Her research interest covers computerized image analysis and molecular probes techniques for biological behavior of breast cancer

    Corresponding author: XU Jun Professor at Nanjing University of Information Science and Technology. His research interest covers computational pathology, digital pathology, computer-aided detection, diagnosis, and prognosis on cancers, deep learning and big data driven analysis for medical data analysis, and clinical transitional medicine. Corresponding author of this paper
  • 摘要: 上皮和间质组织是乳腺组织病理图像中最基本的两种组织,约80%的乳腺肿瘤起源于乳腺上皮组织.为了构建基于乳腺组织病理图像分析的计算机辅助诊断系统和分析肿瘤微环境,上皮和间质组织的自动分割是重要的前提条件.本文构建一种基于逐像素点深度卷积网络(CN-PI)模型的上皮和间质组织的自动分割方法.1)以病理医生标注的两类区域边界附近具有类信息为标签的像素点为中心,构建包含该像素点上下文信息的正方形图像块的训练集.2)以每个正方形图像块包含的像素的彩色灰度值作为特征,以这些图像块中心像素类信息为标签训练CN模型.在测试阶段,在待分割的组织病理图像上逐像素点地取包含每个中心像素点上下文信息的正方形图像块,并输入到预先训练好的CN网络模型,以预测该图像块中心像素点的类信息.3)以每个图像块中心像素为基础,逐像素地遍历图像中的每一个像素,将预测结果作为该图像块中心像素点类信息的预测标签,实现对整幅图像的逐像素分割.实验表明,本文提出的CN-PI模型的性能比基于图像块分割的CN网络(CN-PA)模型表现出了更优越的性能.
    1)  本文责任编委 张道强
  • 图  1  显微镜不同物镜放大倍数下的乳腺肿瘤组织病理图像

    Fig.  1  Histopathological images of breast tumors under different magnification of objective microscope

    图  2  上皮组织的不同形态

    Fig.  2  Different forms of epithelial tissue

    图  3  不同组织病理学分级的图像

    Fig.  3  Images of different histopathological grades

    图  4  本文使用的CN网络结构图

    Fig.  4  The CN net work structure in this paper

    图  5  在边缘处提取训练集小块示意图

    Fig.  5  The images of extracting small block in training set at the edge

    图  6  分割流程图

    Fig.  6  The images of segmentation process

    图  7  定性的分割结果对比

    Fig.  7  Compare in qualitative segmentation results

    图  8  本文模型与对比模型在NKI (a)和VGH (b)数据库中分割结果的ROC曲线

    Fig.  8  The ROC curves of segmentation results in database NKI (a) and VGH (b) of our model and comparison models

    表  1  本文使用的缩写符号及其描述

    Table  1  Abbreviated symbols and their meanings in this paper

    符号 解释
    CN 深度卷积神经网络
    LBP 局部区域二值化
    SVM 支持向量机
    PA 逐图像块
    PI 逐像素
    $ R^e$ 上皮组织图像块
    $ R^s$ 间质组织图像块
    SLIC 简单线性迭代聚类算法
    Ncut 标准化图割算法
    SMC Softmax分类器
    EP 上皮组织
    ST 间质组织
    TMAs 肿瘤组织芯片
    IHC 免疫组织化
    H & E 苏木精和伊红(染色)
    DL 深度学习
    NKI 荷兰癌症数据所数据集
    VGH 温哥华总医院数据集
    ReLU 线性纠正函数
    LRN 局部响应归一化层
    TP 真阳性
    FP 假阳性
    FN 假阴性
    TN 真阴性
    TPR 真阳性率
    TNR 真阴性率
    PPV 阳性预测值
    NPV 阴性预测值
    FPR 假阳性率
    FNR 假阴性率
    FDR 伪发现率
    ACC 准确率
    F1 F1值
    MCC 马修斯相关系数
    CT 平均每张图像的计算时间
    ROC 试者工作特征曲线
    AUC ROC曲线下面积
    下载: 导出CSV

    表  2  训练和测试样本的数量

    Table  2  The number of training and testing samples

    数据集 图像总量 组织 训练图像 测试图像
    图像数量 训练集 验证集 图像数量
    NKI 106 上皮 85 77 804 41 721 21
    间质 70 215 37 625
    VGH 51 上皮 41 40 593 16 914 10
    间质 36 634 15 264
    下载: 导出CSV

    表  3  本文使用的深度卷积网络结构参数

    Table  3  The parameters of deep convolution network structure in this paper

    层数 操作 通道数 尺寸 步长 边缘填充 激活函数 局部归一化
    1 输入 3 - - - -
    2 卷积 32 5 1 2 - -
    3 池化 32 3 2 0 ReLU LRN
    4 卷积 32 5 1 2 ReLU -
    5 池化 32 3 2 0 - LRN
    6 卷积 64 5 1 2 ReLU -
    7 池化 64 3 2 0 - -
    8 全连接 64 - - - - -
    9 全连接 64 - - - - -
    10 输出 2 - - - - -
    下载: 导出CSV

    表  4  本文中不同的对比模型及其描述

    Table  4  Different contrast models and their descriptions in this paper

    模型 图像块生成方法 图像块尺寸 步长(像素) 网络结构 分类器 预测结果
    CN-PA DCNN-SW-SVM[6] 滑动窗+正方形图像块 50×50 25 AlexNet SVM
    DCNN-SW-SMC[6] 滑动窗+正方形图像块 50×50 25 AlexNet SMC 整个图像
    CN-Ncut-SVM[6] 规范化图割+正方形图像块 50×50 AlexNet SVM 区域内像
    CN-Ncut-SMC[6] 规范化图割+正方形图像块 50×50 AlexNet SMC 素类信息
    CN-SLIC-SVM[6] 简单线性迭代聚类+正方形图像块 50×50 AlexNet SVM 相同
    CN-SLIC-SMC[6] 简单线性迭代聚类+正方形图像块 50×50 AlexNet SMC
    CN-PI 滑动窗+正方形图像块 32×32 1 AlexNet SMC 中心像素类信息
    下载: 导出CSV

    表  5  不同模型分割结果的定量评估(%)

    Table  5  Quantitative evaluation of segmentation results for different models (%)

    评估指标
    模型 数据集 TPR TNR PPV NPV FPR FDR FNR ACC F1 MCC CT(s)
    CN-PA CN-SW-SVM NKI 70.40 93.87 92.63 74.33 6.13 7.37 29.60 81.60 80.00 65.60 6
    VGH 87.01 82.20 85.15 84.36 17.80 14.85 12.99 84.79 86.07 69.36 6
    CN-SW-SMC NKI 77.95 80.68 81.63 76.86 19.32 18.37 22.05 79.25 79.25 58.56 3
    VGH 82.18 86.12 87.46 80.40 13.88 12.54 17.82 83.99 84.74 68.08 3
    CN-Ncut-SVM NKI 81.09 86.39 86.72 80.67 13.61 13.27 18.91 83.62 83.81 67.43 1 265
    VGH 88.29 88.40 89.93 86.55 11.60 10.07 11.71 88.34 89.10 76.59 1 265
    CN-Ncut-SMC NKI 88.92 67.94 75.23 84.85 32.06 24.77 11.08 78.91 81.05 58.45 1 262
    VGH 89.37 86.63 88.67 87.42 13.37 11.31 10.63 88.11 89.03 76.05 1 262
    CN-SLIC-SVM NKI 80.63 85.79 86.13 80.18 14.21 13.87 19.37 83.09 83.29 66.37 30
    VGH 88.51 83.68 86.41 86.13 16.32 13.59 11.49 86.28 87.45 72.36 30
    CN-SLIC-SMC NKI 86.31 82.15 84.11 84.60 17.85 15.89 13.66 84.34 85.21 68.60 26
    VGH 87.88 82.13 85.22 85.25 17.87 14.78 12.12 85.23 86.53 70.24 26
    CN-PI NKI 91.05 89.54 90.90 89.71 10.46 9.10 8.95 90.34 90.97 80.59 1 742
    VGH 95.44 93.41 91.95 96.29 6.59 8.06 4.56 94.30 93.66 88.54 1 742
    下载: 导出CSV
  • [1] Chen W Q, Zheng R S, Baade P D, Zhang S W, Zeng H M, Bray F, Jemal A, Yu X Q, He J. Cancer statistics in China, 2015. CA:A Cancer Journal for Clinicians, 2016, 66(2):115-132 doi: 10.3322/caac.21338
    [2] Siegel R L, Miller K D, Jemal A. Cancer statistics, 2015. CA:A Cancer Journal for Clinicians, 2015, 65(1):5-29 doi: 10.3322/caac.21254
    [3] Rubin R, Strayer D S. Rubin's Pathology:Clinicopathologic Foundations of Medicine (5th Edition). Baltimore, MD:Lippincott Williams and Wilkins, 2008.
    [4] Downey C L, Simpkins S A, White J, Holliday D L, Jones J L, Jordan L B, Kulka J, Pollock S, Rajan S S, Thygesen H H, Hanby A M, Speirs V. The prognostic significance of tumour-stroma ratio in oestrogen receptor-positive breast cancer. British Journal of Cancer, 2014, 110(7):1744-1747 doi: 10.1038/bjc.2014.69
    [5] Yuan Y Y, Failmezger H, Rueda O M, Ali H R, Gräf S, Chin S F, Schwarz R F, Curtis C, Dunning M J, Bardwell H, Johnson N, Doyle S, Turashvili G, Provenzano E, Aparicio S, Caldas C, Markowetz F. Quantitative image analysis of cellular heterogeneity in breast tumors complements genomic profiling. Science Translational Medicine, 2012, 4(157):157ra143 http://test.europepmc.org/abstract/MED/23100629
    [6] Xu J, Luo X F, Wang G H, Gilmore H, Madabhushi A. A deep convolutional neural network for segmenting and classifying epithelial and stromal regions in histopathological images. Neurocomputing, 2016, 191:214-223 doi: 10.1016/j.neucom.2016.01.034
    [7] Linder N, Konsti J, Turkki R, Rahtu E, Lundin M, Nordling S, Haglund C, Ahonen T, Pietikäinen M, Lundin J. Identification of tumor epithelium and stroma in tissue microarrays using texture analysis. Diagnostic Pathology, 2012, 7:22 doi: 10.1186/1746-1596-7-22
    [8] Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987 doi: 10.1109/TPAMI.2002.1017623
    [9] Vapnik V N. Statistical Learning Theory. New York:Wiley, 1998.
    [10] Dalal N, Triggs B. Histograms of oriented gradients for human detection. In:Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA, USA:IEEE, 2005. 886-893 http://dl.acm.org/citation.cfm?id=1068507.1069007
    [11] Beck A H, Sangoi A R, Leung S, Marinelli R J, Nielsen T O, van de Vijver M J, West R B, van de Rijn M, Koller D. Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Science Translational Medicine, 2011, 3(108):108ra113 http://europepmc.org/abstract/MED/22072638
    [12] Bourzac K. Software:the computer will see you now. Nature, 2013, 502(7473):S92-S94 doi: 10.1038/502S92a
    [13] Ren X, Malik J. Learning a classification model for segmentation. In:Proceedings of the 9th IEEE International Conference on Computer Vision. Nice, France:IEEE, 2003. 10-17 http://dl.acm.org/citation.cfm?id=946677
    [14] Ali S, Lewis J, Madabhushi A. Spatially aware cell cluster (SpACCl) graphs:predicting outcome in oropharyngeal pl6+ tumors. In:Medical Image Computing and Computer-assisted Intervention:MICCAI International Conference on Medical Image Computing and Computer-assisted Intervention. Berlin Heidelberg:Springer, 2013. 412-419 http://europepmc.org/abstract/MED/24505693
    [15] Hiary H, Alomari R S, Saadah M, Chaudhary V. Automated segmentation of stromal tissue in histology images using a voting Bayesian model. Signal, Image and Video Processing, 2012, 7(6):1229-1237 doi: 10.1007/s11760-012-0393-2
    [16] Eramian M, Daley M, Neilson D, Daley T. Segmentation of epithelium in H&E stained odontogenic cysts. Journal of Microscopy, 2011, 244(3):273-292 doi: 10.1111/jmi.2011.244.issue-3
    [17] Amaral T, McKenna S, Robertson K, Thompson A. Classification and immunohistochemical scoring of breast tissue microarray spots. IEEE Transactions on Biomedical Engineering, 2013, 60(10):2806-2814 doi: 10.1109/TBME.2013.2264871
    [18] Bejnordi B E, Balkenhol M, Litjens G, Holland R, Bult P, Karssemeijer N, van der Laak J A W M. Automated detection of DCIS in whole-slide H&E stained breast histopathology images. IEEE Transactions on Medical Imaging, 2016, 35(9):2141-2150 doi: 10.1109/TMI.2016.2550620
    [19] Cirećsan D C, Giusti A, Gambardella L M, Schmidhuber J. Mitosis detection in breast cancer histology images with deep neural networks. In:Medical Image Computing and Computer-assisted Intervention:MICCAI International Conference on Medical Image Computing and Computer-assisted Intervention. Berlin Heidelberg:Springer, 2013. 411-418 doi: 10.1007/978-3-642-40763-5_51
    [20] Lécun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11):2278-2324 doi: 10.1109/5.726791
    [21] Serre T, Wolf L, Poggio T. Object recognition with features inspired by visual cortex. In:Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA, USA:IEEE, 2005. 994-1000 http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=1467551
    [22] Cruz-Roa A A, Ovalle J E A, Madabhushi A, Osorio F A G. A deep learning architecture for image representation, visual interpretability and automated basal-cell carcinoma cancer detection. In:Medical Image Computing and Computer-assisted Intervention:MICCAI International Conference on Medical Image Computing and Computer-assisted Intervention. Berlin Heidelberg:Springer, 2013. 403-410 doi: 10.1007/978-3-642-40763-5_50
    [23] Su H, Xing F, Kong X F, Xie Y P, Zhang S T, Yang L. Robust cell detection and segmentation in histopathological images using sparse reconstruction and stacked denoising autoencoders. Medical Image Computing and Computer-Assisted Intervention——MICCAI 2015. Switzerland:Springer International Publishing, 2015. 383-390 doi: 10.1007/978-3-319-24574-4_46
    [24] Xu J, Xiang L, Liu Q S, Gilmore H, Wu J Z, Tang J H, Madabhushi A. Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Transactions on Medical Imaging, 2016, 35(1):119-130 doi: 10.1109/TMI.2015.2458702
    [25] Wang H B, Cruz-Roa A, Basavanhally A, Gilmore H, Shih N, Feldman M, Tomaszewski J, Gonzalez F, Madabhushi A. Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features. Journal of Medical Imaging, 2014, 1(3):034003 doi: 10.1117/1.JMI.1.3.034003
    [26] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks. In:Advances in Neural Information Processing Systems 25:Proceedings of the 2012 Conference. Cambridge:MIT Press, 2012. http://dl.acm.org/citation.cfm?id=3065386
    [27] Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. Journal of Machine Learning Research, 2010, 15:315-323 https://hal.archives-ouvertes.fr/hal-00752497
    [28] Reinhard E, Adhikhmin M, Gooch B, Shirley P. Color transfer between images. IEEE Computer Graphics and Applications, 2001, 21(5):34-41 http://dl.acm.org/citation.cfm?id=618848
    [29] Jia Y Q, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T. Caffe:convolutional architecture for fast feature embedding. Eprint Arxiv, 2014. arXiv:1408.5093v1 http://dl.acm.org/citation.cfm?id=2654889&preflayout=flat
    [30] Schölkopf B, Platt J, Hofmann T. Greedy layerwise training of deep networks. In:Advances in Neural Information Processing Systems 19:Proceedings of the 2006 Conference. Cambridge:MIT Press, 2007. 153-160
  • 加载中
图(8) / 表(5)
计量
  • 文章访问数:  3559
  • HTML全文浏览量:  321
  • PDF下载量:  611
  • 被引次数: 0
出版历程
  • 收稿日期:  2016-06-13
  • 录用日期:  2016-11-08
  • 刊出日期:  2017-11-20

目录

    /

    返回文章
    返回