2.624

2020影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

中医舌象分割技术研究进展: 方法、性能与展望

卢运西 李晓光 张辉 张菁 卓力

卢运西, 李晓光, 张辉, 张菁, 卓力. 中医舌象分割技术研究进展: 方法、性能与展望.自动化学报, 2021, 47(5): 1005-1016 doi: 10.16383/j.aas.c180807
引用本文: 卢运西, 李晓光, 张辉, 张菁, 卓力. 中医舌象分割技术研究进展: 方法、性能与展望.自动化学报, 2021, 47(5): 1005-1016 doi: 10.16383/j.aas.c180807
Lu Yun-Xi, Li Xiao-Guang, Zhang Hui, Zhang Jing, Zhuo Li. Review on tongue image segmentation technologies for traditional Chinese medicine: methodologies, performances and prospects. Acta Automatica Sinica, 2021, 47(5): 1005-1016 doi: 10.16383/j.aas.c180807
Citation: Lu Yun-Xi, Li Xiao-Guang, Zhang Hui, Zhang Jing, Zhuo Li. Review on tongue image segmentation technologies for traditional Chinese medicine: methodologies, performances and prospects. Acta Automatica Sinica, 2021, 47(5): 1005-1016 doi: 10.16383/j.aas.c180807

中医舌象分割技术研究进展: 方法、性能与展望

doi: 10.16383/j.aas.c180807
基金项目: 

国家自然科学基金面上项目 61871006

详细信息
    作者简介:

    卢运西  北京工业大学计算机技术专业硕士研究生. 2015年获得北京工商大学信息工程系学士学位. 主要研究方向为图像处理和模式识别. E-mail: yunxilu@emails.bjut.edu.cn

    张辉  北京工业大学信息学部讲师. 2010年获得北京理工大学信号与信息处理专业博士学位. 主要研究方向为计算机视觉, 机器学习在多媒体内容分析、视觉追踪、目标检测中的应用. E-mail: huizhang@bjut.edu.cn

    张菁  北京工业大学教授. 2008年获得北京工业大学博士学位. 美国德州大学圣安东尼奥分校计算机科学系访问学者. 主要研究方向为图像处理, 图像识别, 图像检索. E-mail: zhj@bjut.edu.cn

    卓力  北京工业大学教授. 1992年获得电子科技大学无线电技术系工学学士学位, 1998年和2004年分别获得东南大学信号与信息处理专业硕士学位和北京工业大学模式识别与智能系统专业博士学位. 主要研究方向为图像/视频编码和传输, 多媒体内容分析, 多媒体信息安全. E-mail: zhuoli@bjut.edu.cn

    通讯作者:

    李晓光  北京工业大学副教授. 2003年于北京工业大学电子与信息工程专业获得学士学位, 2008年获得北京工业大学博士学位. 主要研究方向为计算机视觉/图像增强, 图像复原. 本文通信作者. E-mail: lxg@bjut.edu.cn

Review on Tongue Image Segmentation Technologies for Traditional Chinese Medicine: Methodologies, Performances and Prospects

Funds: 

National Natural Science Foundation of China 61871006

More Information
    Author Bio:

    LU Yun-Xi  Master student in computer science and technology at Beijing University of Technology. He received his bachelor degree from the Department of information engineering at Beijing Technology and Business University in 2015. His research interest covers image processing and pattern recognition

    ZHANG Hui  Lecturer at the Faculty of Information, Beijing University of Technology. He received his Ph. D. degree in signal and information processing from Beijing Institute of Technology in 2010. His research interest covers computer vision, and machine learning techniques applied to multimedia content analysis, visual tracking and object detection

    ZHANG Jing  Professor at Beijing University of Technology, visiting scholar in the Department of Computer Science, University of Texas at San Antonio, USA. She received her Ph. D. degree from Beijing University of Technology in 2008. Her research interest covers image processing, image recognition, and image retrieval

    ZHUO Li  Professor at Beijing University of Technology. She received her bachelor degree in radio technology from University of Electronic Science and Technology in 1992, master degree in signal and information processing from Southeast University in 1998, and Ph. D. degree in pattern recognition and intellectual system from Beijing University of Technology in 2004. Her research interest covers image/video coding and transmission, multimedia content analysis, and multimedia information security

    Corresponding author: LI Xiao-Guang  Associate professor at Beijing University of Technology. He received his bachelor degree in electronic and information engineering from Beijing University of Technology in 2003. He received his Ph. D. degree from Beijing University of Technology in 2008. His research interest covers computer vision, image enhancement, and image restoration. Corresponding author of this paper
  • 摘要: 中医舌诊的客观化、定量化研究是中医现代化发展中的重要课题. 数字化采集到的舌图像包括舌体及部分面部区域, 为了便于后续舌象自动分析, 需要首先将舌体部分从图像中分割出来, 分割效果将直接影响后续舌象特征分析的准确性. 基于传统方法的舌象分割技术虽然取得了很大进展, 但其性能仅能达到半自动分割, 对较难分割的图像往往需要借助人机交互来完成. 近年来, 深度学习技术在图像处理及计算机视觉等多个领域取得了突破, 其在图像语义分割任务中也取得了远超传统方法的进展. 基于深度学习的舌象分割技术已经基本实现了全自动的鲁棒分割. 本文首先从传统分割方法和基于深度学习的分割方法两方面对中医舌象分割技术发展中的主要方法进行综述; 其次, 采用我们收集的舌象数据库对典型的方法进行性能评估, 并对不同舌象分割方法的特点进行分析与讨论. 最后, 对中医舌图像分割方法潜在的发展方向进行了展望.
    Recommended by Associate Editor LIU Cheng-Lin
    1)  本文责任编委 刘成林
  • 图  1  几种基于图像特征的舌体分割方法分割效果

    Fig.  1  The results of several traditional algorithms

    图  2  卷积神经网络卷积化

    Fig.  2  Convolutionalization of CNN

    图  3  全卷积网络结构图

    Fig.  3  The architecture of FCN

    图  4  SegNet网络结构图

    Fig.  4  The architechture of SegNet

    图  5  基于Atrous卷积的空间金字塔池化结构图

    Fig.  5  The architechture of ASPP

    图  6  改进的基于Atrous卷积的空间金字塔池化结构图

    Fig.  6  The architechture of the improved ASPP

    图  7  Tongue dataset中的部分舌图像

    Fig.  7  Some pictures of the tongue dataset

    图  8  测试舌图像(彩色图像见网络版)

    Fig.  8  Pictures of test dataset (Refer to the internet version for color images

    图  9  不同分割算法的分割效果(彩色图像见网络版)

    Fig.  9  The results of diffierent segmentation algorithms (Refer to the internet version for color images)

    表  1  不同舌象分割方法性能比较

    Table  1  Comparison on performances of diffierent algorithms

    网络名称mIoU预测时间(s)
    FCN8S0.83227.4227
    FCN16S0.87187.4403
    FCN32S0.92727.5273
    SegNet0.92770.0014
    Mask R-CNN0.93610.1412
    Deeplab V20.96780.0021
    DeeplabV30.98104.9999×10-6
    DeeplabV3+0.98182.9999×10-6
    下载: 导出CSV
  • [1] 沈兰荪, 蔡轶珩, 张新峰. 中医舌象的采集与分析. 北京: 北京工业大学出版社, 2007.

    Shen Lan-Sun, Cai Yi-Heng, Zhang Xin-Feng. Capturation and Analysis of Chinese Medicine Tongue Images. Beijing: Beijing University of Technology Press, 2007.
    [2] 李国正, 史淼晶, 李福凤, 王忆勤. 舌体图像分割技术的实验分析与改进. 山东大学学报(工学版), 2010, 40(5): 87-95 https://www.cnki.com.cn/Article/CJFDTOTAL-SDGY201005017.htm

    Li Guo-Zheng, Shi Miao-Jing, Li Fu-Feng, Wang Yi-Qin. An empirical study on tongue image detection. Journal of Shandong University (Engineering Science), 2010, 40(5): 87-95 https://www.cnki.com.cn/Article/CJFDTOTAL-SDGY201005017.htm
    [3] Kim K H, Do J H, Ryu H, Kim J Y. Tongue diagnosis method for extraction of effective region and classification of tongue coating. In: Proceedings of the 1st Workshops on Image Processing Theory, Tools and Applications. Sousse, Tunisia: IEEE, 2008. 1-7
    [4] 邬文隽, 马利庄, 肖学中. 一种结合亮度和粗糙度信息的舌像分割方法. 系统仿真学报, 2006, 18(S1): 374-376, 379 https://www.cnki.com.cn/Article/CJFDTOTAL-XTFZ2006S1115.htm

    Wu Wen-Jun, Ma Li-Zhuang, Xiao Xue-Zhong. Method of tongue image segmentation based on luminance and roughness information. Journal of System Simulation, 2006, 18(S1): 374-376, 379 https://www.cnki.com.cn/Article/CJFDTOTAL-XTFZ2006S1115.htm
    [5] 张志顺, 刘勇. 基于动态阈值和修正模型的舌体提取算法. 计算机与现代化, 2014, (11): 49-52 doi: 10.3969/j.issn.1006-2475.2014.11.011

    Zhang Zhi-Shun, Liu Yong. Tongue extraction algorithm based on dynamic thresholds and correction model. Computer and Modernization, 2014, (11): 49-52 doi: 10.3969/j.issn.1006-2475.2014.11.011
    [6] 赵忠旭, 王爱民, 沈兰荪. 基于数学形态学和HIS模型的彩色舌图像分割. 北京工业大学学报, 1999, 25(2): 67-71 https://www.cnki.com.cn/Article/CJFDTOTAL-BJGD199902011.htm

    Zhao Zhong-Xu, Wang Ai-Min, Shen Lan-Sun. The color tongue image segmentation based on mathematical morphology and HIS model. Journal of Beijing Polytechnic University, 1999, 25(2): 67-71 https://www.cnki.com.cn/Article/CJFDTOTAL-BJGD199902011.htm
    [7] Chen L, Wang D Y, Liu Y Q, Gao X H, Shang H L. A novel automatic tongue image segmentation algorithm: Color enhancement method based on L*a*b* color space. In: Proceedings of the 2015 IEEE International Conference on Bioinformatics and Biomedicine. Washington, DC, USA: IEEE, 2015. 990-993
    [8] 李丹霞, 韦玉科. 基于自适应阈值的舌像分割方法. 计算机技术与发展, 2011, 21(9): 63-65 doi: 10.3969/j.issn.1673-629X.2011.09.016

    Li Dan-Xia, Wei Yu-Ke. Tongue image segmentation method based on adaptive thresholds. Computer Technology and Development, 2011, 21(9): 63-65 doi: 10.3969/j.issn.1673-629X.2011.09.016
    [9] 蒋依吾, 陈建仲, 张恒鸿. 电脑化中医舌诊系统. 中国中西医结合杂志, 2000, 20(2): 145-147 doi: 10.3321/j.issn:1003-5370.2000.02.023

    Jiang Yi-Wu, Chen Jian-Zhong, Zhang Heng-Hong. Computerized Chinese medicine tongue diagnosis system. Chinese Journal of Integrated Traditional and Western Medicine, 2000, 20(2): 145-147 doi: 10.3321/j.issn:1003-5370.2000.02.023
    [10] Zhi L, Yan J Q, Zhou T, Tang Q L. Tongue shape detection based on b-spline. In: Proceedings of the 2006 International Conference on Machine Learning and Cybernetics. Dalian, China: IEEE, 2006. 3829-3832
    [11] Yu S Y, Yang J, Wang Y G, Zhang Y. Color active contour models based tongue segmentation in traditional Chinese medicine. In: Proceedings of the 1st International Conference on Bioinformatics and Biomedical Engineering. Wuhan, China: IEEE, 2007. 1065-1068
    [12] Li X, Li J, Wang D. Automatic tongue image segmentation based on histogram projection and matting. In: Proceedings of the 2014 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2014: 76-81
    [13] Fu Z C, Li W, Li X Q, Li F F, Wang Y Q. Automatic tongue location and segmentation. In: Proceedings of the 2008 International Conference on Audio, Language, and Image Processing. Shanghai, China: IEEE, 2008. 1050-1055
    [14] Xie T, Xia C M, Chen F F, Zhang S L, Zhang Y. A method of tongue image segmentation based on kernel FCM. In: Proceedings of the 9th International Congress on Image and Signal Processing, BioMedical Engineering, and Informatics. Datong, China: IEEE, 2016. 319-324
    [15] Liu Z, Yan J Q, Zhang D, Li Q L. Automated tongue segmentation in hyperspectral images for medicine. Applied Optics, 2007, 46(34): 8328-8334 doi: 10.1364/AO.46.008328
    [16] Li X Q, Li J D, Wang D. Automatic tongue image segmentation based on histogram projection and matting. In: Proceedings of the 2014 IEEE International Conference on Bioinformatics and Biomedicine. Belfast, UK: IEEE, 2014. 76-81
    [17] Xu W T, Kanawong R, Xu D, Li S, Ma T, Zhang G X, et al. An automatic tongue detection and segmentation framework for computer-aided tongue image analysis. In: Proceedings of the 13th IEEE International Conference on e-Health Networking, Applications and Services. Columbia, MO, USA: IEEE, 2011. 189-192
    [18] Pang B, Zhang D, Wang K Q. The bi-elliptical deformable contour and its application to automated tongue segmentation in Chinese medicine. IEEE Transactions on Medical Imaging, 2005, 24(8): 946-956 doi: 10.1109/TMI.2005.850552
    [19] Shi M J, Li G Z, Li F F, Xu C. Computerized tongue image segmentation via the double geo-vector flow. Chinese Medicine, 2014, 9: Article No. 7 doi: 10.1186/1749-8546-9-7
    [20] Kass M, Witkin A, Terzopoulos D. Snakes: Active contour models. International Journal of Computer Vision, 1988, 1(4): 321-331 doi: 10.1007/BF00133570
    [21] Kang D J. A fast and stable snake algorithm for medical images. Pattern Recognition Letters, 1999, 20(5): 507-512 doi: 10.1016/S0167-8655(99)00019-7
    [22] Luo H, Gaborski R, Acharya R. Automatic segmentation of lung regions in chest radiographs: A model guided approach. In: Proceedings of the 2000 International Conference on Image Processing. Vancouver, BC, Canada: IEEE, 2000. 483-486
    [23] McInerney T, Terzopoulos D. Deformable models in medical image analysis: A survey. Medical Image Analysis, 1996, 1(2): 91-108 doi: 10.1016/S1361-8415(96)80007-7
    [24] 王爱民, 沈兰荪. 图像分割研究综述. 测控技术, 2000, 19(5): 1-6, 16 doi: 10.3969/j.issn.1000-8829.2000.05.001

    Wang Ai-Min, Shen Lan-Sun. Study surveys on image segmentation. Measurement and Control Technology, 2000, 19(5): 1-6, 16 doi: 10.3969/j.issn.1000-8829.2000.05.001
    [25] 沈兰荪, 王爱民. 图像分割的分级性研究. 北京工业大学学报, 2000, 26(3): 33-38 doi: 10.3969/j.issn.0254-0037.2000.03.007

    Shen Lan-Sun, Wang Ai-Min. Hierarchy of image segmentation. Journal of Beijing Polytechnic University, 2000, 26(3): 33-38 doi: 10.3969/j.issn.0254-0037.2000.03.007
    [26] McInerney T, Terzopoulos D. Topologically adaptable snakes. In: Proceedings of the 1995 IEEE International Conference on Computer Vision. Cambridge, MA, USA: IEEE, 1995. 840-845
    [27] 王艳清, 卫保国, 蔡轶珩, 沈兰荪. 一种基于先验知识的自动舌体分割算法. 电子学报, 2004, 32(3): 489-491 doi: 10.3321/j.issn:0372-2112.2004.03.031

    Wang Yan-Qing, Wei Bao-Guo, Cai Yi-Heng, Shen Lan-Sun. A knowledge-based arithmetic for automatic tongue segmentation. Acta Electronica Sinica, 2004, 32(3): 489-491 doi: 10.3321/j.issn:0372-2112.2004.03.031
    [28] 王郁中, 杨杰, 周越, 郑元杰, 王忆勤. 图像分割技术在中医舌诊客观化研究中的应用. 生物医学工程学杂志, 2005, 22(6): 1128- 1133 doi: 10.3321/j.issn:1001-5515.2005.06.011

    Wang Yu-Zhong, Yang Jie, Zhou Yue, Zheng Yuan-Jie, Wang Yi-Qin. Image segmentation in tongue characterization. Journal of Biomedical Engineering, 2005, 22(6): 1128- 1133 doi: 10.3321/j.issn:1001-5515.2005.06.011
    [29] 吴佳, 张永红, 白净, 翁维良, 吴煜, 韩垚, 等. 基于分水岭变换和主动轮廓模型的舌体轮廓图像分割. 清华大学学报(自然科学版), 2008, 48(6): 1040-1043 doi: 10.3321/j.issn:1000-0054.2008.06.032

    Wu Jia, Zhang Yong-Hong, Bai Jing, Weng Wei-Liang, Wu Yu, Han Yao, et al. Tongue contour image extraction using a watershed transform and an active contour model. Journal of Tsinghua University (Science and Technology), 2008, 48(6): 1040-1043 doi: 10.3321/j.issn:1000-0054.2008.06.032
    [30] Zhai X M, Lu H D, Zhang L Z. Application of image segmentation technique in tongue diagnosis. In: Proceedings of the 2009 International Forum on Information Technology and Applications. Chengdu, China: IEEE, 2009. 768-771
    [31] 覃武星, 李斌, 岳小强. 一种基于初始化Snake轮廓线的混合舌图像分割算法. 中国科学技术大学学报, 2010, 40(8): 807-811 doi: 10.3969/j.issn.0253-2778.2010.08.007

    Qin Wu-Xing, Li Bin, Yue Xiao-Qiang. A hybrid tongue image segmentation algorithm based on initialization of snake contours. Journal of University of Science and Technology of China, 2010, 40(8): 807-811 doi: 10.3969/j.issn.0253-2778.2010.08.007
    [32] Pang B, Wang K Q, Zhang S, Zhang F M. On automated tongue image segmentation in Chinese medicine. In: Proceedings of the 2002 Object Recognition Supported by User Interaction for Service Robots. Quebec City, Quebec, Canada: IEEE, 2002. 616-619
    [33] Guo J W, Yang Y K, Wu Q W, Su J L, Ma F. Adaptive active contour model based automatic tongue image segmentation. In: Proceedings of the 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics. Datong, China: IEEE, 2016. 1386-1390
    [34] 孙晓琳, 庞春颖. 一种改进snake模型的中医舌体分割方法. 长春理工大学学报(自然科学版), 2013, 36(5): 154-156 doi: 10.3969/j.issn.1672-9870.2013.05.043

    Sun Xiao-Lin, Pang Chun-Ying. An improved snake model method on tongue segmentation. Journal of Changchun University of Science and Technology (Natural Science Edition), 2013, 36(5): 154-156 doi: 10.3969/j.issn.1672-9870.2013.05.043
    [35] 王明英, 张新峰, 卓力. 一种基于Snakes模型的中医舌象分割改进算法. 测控技术, 2011, 30(5): 32-35 doi: 10.3969/j.issn.1000-8829.2011.05.007

    Wang Ming-Ying, Zhang Xin-Feng, Zhuo Li. An improved snakes model for tongue image segmentation. Measurement and Control Technology, 2011, 30(5): 32-35 doi: 10.3969/j.issn.1000-8829.2011.05.007
    [36] Zhang H Z, Zuo W M, Wang K Q, Zhang D. A snake-based approach to automated segmentation of tongue image using polar edge detector. International Journal of Imaging Systems and Technology, 2006, 16(4): 103-112 doi: 10.1002/ima.20075
    [37] Salakhutdinov R, Mnih A, Hinton G. Restricted Boltzmann machines for collaborative filtering. In: Proceedings of the 24th International Conference on Machine Learning. Corvalis, Oregon, USA: ACM, 2007. 791-798
    [38] Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640 -651 doi: 10.1109/TPAMI.2016.2572683
    [39] He K M, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 386-397 doi: 10.1109/TPAMI.2018.2844175
    [40] Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149 doi: 10.1109/TPAMI.2016.2577031
    [41] Badrinarayanan V, Kendall A, Cipolla R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12): 2481-2495
    [42] Qu P L, Zhang H, Zhuo L, Zhang J, Chen G Y. Automatic tongue image segmentation for traditional Chinese medicine using deep neural network. In: Proceedings of the 13th International Conference on Intelligent Computing. Liverpool, UK: Springer, 2017. 247-259
    [43] Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer, 2015. 234-241
    [44] Chen L C, Zhu Y K, Papandreou G, Schroff F, Adam H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: Springer, 2018. 833-851
    [45] Chen L C, Papandreou G, Kokkinos I, Murphy K, Yuille A L. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848 doi: 10.1109/TPAMI.2017.2699184
    [46] Zhao H S, Shi J P, Qi X J, Wang X G, Jia J Y. Pyramid scene parsing network. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017. 6230-6239
    [47] Chen L C, Papandreou G, Schroff F, Adam H. Rethinking atrous convolution for semantic image segmentation. arXiv: 1706.05587, 2017.
    [48] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016. 770-778
    [49] He K M, Zhang X Y, Ren S Q, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland: Springer, 2014. 346-361
    [50] Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning. Lille, France: JMLR. org, 2015. 448-456
    [51] Zhang D, Zhang H Z, Zhang B. Tongue Image Analysis. Singapore: Springer, 2017.
    [52] Huang Q, Dom B. Quantitative methods of evaluating image segmentation. In: Proceedings of the 1995 International Conference on Image Processing. Washington, DC, USA: IEEE, 1995. 53-56
  • 加载中
图(9) / 表(1)
计量
  • 文章访问数:  481
  • HTML全文浏览量:  204
  • PDF下载量:  212
  • 被引次数: 0
出版历程
  • 收稿日期:  2018-12-05
  • 录用日期:  2019-02-13
  • 刊出日期:  2021-05-21

目录

    /

    返回文章
    返回