2.793

2018影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于深度学习的纹理布匹瑕疵检测方法

许玉格 钟铭 吴宗泽 任志刚 刘伟生

许玉格, 钟铭, 吴宗泽, 任志刚, 刘伟生. 基于深度学习的纹理布匹瑕疵检测方法. 自动化学报, 2020, 45(x): 1−15 doi: 10.16383/j.aas.c200148
引用本文: 许玉格, 钟铭, 吴宗泽, 任志刚, 刘伟生. 基于深度学习的纹理布匹瑕疵检测方法. 自动化学报, 2020, 45(x): 1−15 doi: 10.16383/j.aas.c200148
Xu Yu-Ge, Zhong Ming, Wu Zong-Ze, Ren Zhi-Gang, Liu Wei-Shen. Detection of detecting textured fabric defects based on deep learning. Acta Automatica Sinica, 2020, 45(x): 1−15 doi: 10.16383/j.aas.c200148
Citation: Xu Yu-Ge, Zhong Ming, Wu Zong-Ze, Ren Zhi-Gang, Liu Wei-Shen. Detection of detecting textured fabric defects based on deep learning. Acta Automatica Sinica, 2020, 45(x): 1−15 doi: 10.16383/j.aas.c200148

基于深度学习的纹理布匹瑕疵检测方法

doi: 10.16383/j.aas.c200148
基金项目: 国家自然科学基金(61703114, 61673126, U1701261, 51675108)资助
详细信息
    作者简介:

    许玉格:华南理工大学自动化科学与工程学院副教授. 主要研究方向为机器学习与智能计算. E-mail: xuyuge@scut.edu.cn

    钟铭:华南理工大学自动化科学与工程学院硕士研究生. 主要研究方向为深度学习, 计算机视觉. E-mail: tdlming@163.com

    吴宗泽:广东工业大学自动化学院教授. 主要研究方向有自动控制, 大数据, 知识自动化, 人工智能. 本文通信作者. E-mail: zzwu@gdut.edu.cn

    任志刚:广东工业大学自动化学院讲师. 主要研究方向有最优控制, 知识自动化, 人工智能. E-mail: renzhigang@gdut.edu.cn

    刘伟生:西安交通大学电信学院硕士. 深圳禾思众成科技有限公司CTO, 主要从事深度学习在工业质检领域的研究. E-mail: liuweisheng1992@outlook.com

Detection of Detecting Textured Fabric Defects Based on Deep Learning

Funds: Supported by the National Natural Science Foundation of China through grants (61673126, 61703114, U1701261, 51675108)
  • 摘要: 布匹瑕疵检测是纺织工业中产品质量评估的关键环节, 实现快速、准确、高效的布匹瑕疵检测对于提升纺织工业的产能具有重要意义. 在实际布匹生产过程中, 布匹瑕疵在形状、大小及数量分布上存在不平衡问题, 且纹理布匹复杂的纹理信息会掩盖瑕疵的特征, 加大布匹瑕疵检测难度. 本文提出基于深度卷积神经网络的分类不平衡纹理布匹瑕疵检测方法, 首先建立一种基于通道叠加的ResNet50卷积神经网络模型(ResNet50+)对布匹瑕疵特征进行优化提取; 其次提出一种冗余特征过滤的特征金字塔网络对特征图中的背景特征进行过滤, 增强其中瑕疵特征的语义信息; 最后构造针对瑕疵数量进行加权的Multi Focal Loss损失函数, 减轻数据集不平衡对模型的影响, 降低模型对于少数类瑕疵的不敏感性. 通过实验对比, 本文提出的方法能有效提升布匹瑕疵检测的准确率及定位精度, 同时降低了布匹瑕疵检测的误检率和漏检率, 明显优于当前主流的布匹瑕疵检测算法.
  • 图  1  纹理布匹瑕疵样本图片

    Fig.  1  Samples of jacquard fabric defects

    图  2  纹理布匹瑕疵形状分布

    Fig.  2  Shape Distribution of jacquard fabric defects

    图  3  ITF-DCNN模型的整体结构图

    Fig.  3  Structure of detection model

    图  4  Resnet50网络结构图

    Fig.  4  Model structure of ResNet50

    图  5  残差模块

    Fig.  5  Model structure of residual block

    图  6  特征图过滤方式

    Fig.  6  Methods to filtering feature maps

    图  7  模型的Loss收敛曲线图

    Fig.  7  The loss curves of models

    图  8  ITF-DCNN模型检测结果图

    Fig.  8  Experimental results of the proposed method ITF-DCNN

    图  9  MFL在少数类上的检测结果图

    Fig.  9  Experimental result of MFL based on minority classes

    图  10  模板图片

    Fig.  10  Template Images

    图  11  FPN和F-FPN的泛化性实验对比图

    Fig.  11  Comparison of FPN and F-FPN generalization experiments

    表  1  增强前后数据集中的样本分布

    Table  1  Samples distribution of the data set before and after data augmentation

    瑕疵类别沾污花毛虫粘破洞蜡斑网折其他正常总计
    训练集增强前243239820812273775227566118
    训练集增强后972815948344902923062061102624476
    验证集增强前14133126254420623
    验证集增强后56213448226221416722490
    下载: 导出CSV

    表  2  数据集增强前后模型准确率对比实验结果

    Table  2  Experimental results of model on accuracy before and after data set enhancement

    瑕疵类别沾污花毛虫粘破洞蜡斑网折其他正常均值
    数据集增强前88.24%83.36%87.56%89.36%83.78%88.21%89.65%98.66%88.61%
    数据集增强后90.56%85.51%90.35%91.42%87.64%89.24%90.02%99.81%90.57%
    下载: 导出CSV

    表  3  数据集增强前后模型mAP对比实验结果

    Table  3  Experimental results of model on mAP before and after data set enhancement

    瑕疵类别沾污花毛虫粘破洞蜡斑网折其他正常均值
    数据集增强前69.06%58.51%81.50%83.44%33.33%63.70%45.51%62.15%
    数据集增强后70.04%59.12%83.23%83.54%35.78%63.70%47.31%63.25%
    下载: 导出CSV

    表  4  不同模型在布匹瑕疵数据集上的实验结果

    Table  4  Experimental results of different models on the jacquard fabric defect data set

    DetectorBackbonemAP准确率误检率漏检率
    Faster R-CNNResnet5065.56%87.40%12.60%1.42%
    Cascade R-CNNResnet5063.77%90.55%9.45%2.85%
    RetinaNetResnet5065.60%53.86%46.13%0.20%
    Faster R-CNNResnet10163.85%88.72%11.28%2.24%
    Cascade R-CNNResnet10164.60%90.35%9.65%1.83%
    RetinaNetResnet10166.52%56.23%43.77%0.12%
    GLCM64.63%35.37%6.87%
    Gabor83.87%16.13%1.67%
    GMM81.32%18.68%1.77%
    PTIT[38]92.56%7.44%0.94%
    CAE-SGAN[41]85.01%14.99%2.65%
    SurfNet[42]84.82%15.18%1.79%
    ITF-DCNNResnet5073.41%97.56%2.44%1.65%
    ITF-DCNNResnet10173.92%97.66%2.34%1.14%
    下载: 导出CSV

    表  9  改进后的ResNet50+网络性能对比实验

    Table  9  Experimental performance result of ResNet50+

    mAP准确率误检率漏检率
    ResNet5063.77%90.55%9.45%2.85%
    ResNet50+I63.68%91.76%8.24%2.66%
    ResNet50+C64.14%92.31%7.69%3.43%
    ResNet50+64.72%92.78%7.22%2.91%
    下载: 导出CSV

    表  5  F-FPN性能验证实验结果

    Table  5  Experimental performance result of F-FPN

    mAP准确率误检率漏检率
    Top-Down FPN63.77%90.55%9.45%2.85%
    PANet65.69%92.23%7.77%2.56%
    加性F-FPN70.31%93.65%6.53%1.95%
    卷积F-FPN71.42%96.72%3.28%1.25%
    下载: 导出CSV

    表  6  Multi Focal Loss的性能验证实验结果

    Table  6  Experimental performance result of Multi Focal Loss

    Loss Function$ \alpha $$ \gamma $$ \omega $mAP准确率误检率漏检率
    CE 63.77% 90.55% 9.45% 2.85%
    FL 0.25 5 52.53% 70.23% 29.77% 9.56%
    FL 0.25 2 65.62% 92.86% 7.14% 2.02%
    FL 0.25 1 64.88% 91.91% 8.09% 2.12%
    FL 0.5 0.5 64.74% 91.55% 8.45% 2.33%
    FL 0.75 0.2 59.27% 83.02% 16.98% 8.50%
    FL 0.75 0.1 58.11% 80.85% 19.15% 7.56%
    FL 0.75 0 58.01% 81.22% 18.78% 7.66%
    MFL 1 0.618 68.21% 94.39% 5.61% 1.44%
    MFL 2 0.618 70.12% 95.32% 4.68% 1.68%
    MFL 5 0.618 68.11% 94.50% 5.50% 1.56%
    MFL 2 0.1 67.22% 93.68% 6.32% 2.26%
    MFL 2 0.3 69.03% 94.88% 5.12% 1.56%
    MFL 2 1.0 69.22% 95.17% 4.83% 1.29%
    MFL 2 2.0 68.81% 94.35% 5.65% 1.68%
    MFL 2 5.0 64.38% 92.41% 7.59% 2.42%
    下载: 导出CSV

    表  7  采用F-FPN的模型在不同模板上的泛化性分析

    Table  7  Generalization Analysis of Models Using F-FPN on Different Templates

    模板1模板2模板3模板4模板5模板6模板7模板8模板9模板10均值
    准确率95.87%96.79%99.67%93.56%91.74%91.11%93.66%99.12%98.23%93.65%95.34%
    mAP69.12%69.73%75.37%68.97%67.46%68.12%68.24%75.96%76.82%68.02%70.78%
    下载: 导出CSV

    表  8  采用FPN的模型在不同模板上的泛化性分析

    Table  8  Generalization Analysis of Models Using FPN on Different Templates

    模板1模板2模板3模板4模板5模板6模板7模板8模板9模板10均值
    准确率91.63%91.04%92.38%90.39%88.34%88.12%91.25%91.75%91.42%89.11%90.54%
    mAP62.43%61.75%65.86%62.01%61.99%61.08%62.51%65.63%66.74%61.46%63.07%
    下载: 导出CSV
  • [1] 张慧, 王坤峰, 王飞跃. 深度学习在目标视觉检测中的应用进展与展望. 自动化学报, 2017, 48(4): 1289−1305

    Zhang Hui, Wang Kun-Feng, Wang Fei-Yue. Advances and perspectives on applications of deep learning in visual object detection. Acta Automatica Sinica, 2017, 48(4): 1289−1305
    [2] Ajay Kumar. Computer-vision-based fabric defect detection: A survey. IEEE Transactions on Industrial Electronics, 2008, 55(1): 348−363 doi: 10.1109/TIE.1930.896476
    [3] Chi-ho Chan and Grantham KH Pang. Fabric defect detection by fourier analysis. IEEE Transactions on Industry Applications, 2000, 36(5): 1267−1276 doi: 10.1109/28.871274
    [4] Kazım Hanbay, Muhammed Fatih Talu, and Ömer Faruk Özgüven. Fabric defect detection systems and methods—a systematic literature review. Optik, 2016, 127(24): 11960−11973 doi: 10.1016/j.ijleo.2016.09.110
    [5] Dmitry Chetverikov and Allan Hanbury. Finding defects in texture using regularity and local orientation. Pattern Recognition, 2002, 35(10): 2165−2180 doi: 10.1016/S0031-3203(01)00188-1
    [6] Henry YT Ngan, Grantham KH Pang, and Nelson HC Yung. Automated fabric defect detection—a review. Image and Vision Computing, 2011, 29(7): 442−458 doi: 10.1016/j.imavis.2011.02.002
    [7] Daniel Yapi, Mohand Saïd Allili, and Nadia Baaziz. Automatic fabric defect detection using learning-based local textural distributions in the contourlet domain. IEEE Transactions on Automation Science and Engineering, 2017, 15(3): 1014−1026
    [8] S Abdolkarim Hosseini Ravandi and Koichiro Toriumi. Fourier transform analysis of plain weave fabric appearance. Textile Research Journal, 1995, 65(11): 676−683 doi: 10.1177/004051759506501108
    [9] Jaume Escofet, Maria Sagrario Millan Garcia-Verela, Hector C Abril, and E Torrecilla. Inspection of fabric resistance to abrasion by fourier analysis. In Optics in Computing'98, volume 3490, pages 207−210. International Society for Optics and Photonics, 1998.
    [10] Ido Zachevsky and Yehoshua Y Zeevi. Model-based color natural stochastic textures processing and classification. In 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pages 1357-1361. IEEE, 2015.
    [11] 李敏, 崔树芹, 谢治平. 高斯混合模型在印花织物疵点检测中的应用. 纺织学报, 2015, 36(08): 94−98

    LI Min and CUI Shuqin and XIE Zhiping. Application of Gaussian mixture model on defect detection of print fabric. Journal of Textile Research, 2015, 36(08): 94−98
    [12] Mohand Said Allili, Nadia Baaziz, and Marouene Mejri. Texture modeling using contourlets and finite mixtures of generalized gaussian distributions and applications. IEEE Transactions on Multimedia, 2014, 16(3): 772−784 doi: 10.1109/TMM.2014.2298832
    [13] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, pages 1097−1105, 2012.
    [14] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409.1556, 2014.
    [15] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2818−2826, 2016.
    [16] Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-First AAAI Conference on Artificial Intelligence, 2017.
    [17] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv: 1502.03167, 2015.
    [18] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1−9, 2015.
    [19] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770−778, 2016.
    [20] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4700−4708, 2017.
    [21] Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delving into high quality object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6154−6162, 2018.
    [22] Mingxing Tan, Ruoming Pang, and Quoc V Le. Efficientdet: Scalable and efficient object detection. arXiv preprint arXiv: 1911.09070, 2019.
    [23] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2117−2125, 2017.
    [24] 周波, 李俊峰. 结合目标检测的人体行为识别. 自动化学报, 2019: 1−11 doi: 10.16383/j.aas.c180848

    Zhou Bo and Li Jun-Feng. Human Action Recognition Combined With Object Detection. Acta Automatica Sinica, 2019: 1−11 doi: 10.16383/j.aas.c180848
    [25] 范家伟, 张如如, 陆萌. 深度学习方法在糖尿病视网膜病变诊断中的应用. 自动化学报, 2019

    FAN Jia-Wei and ZHANG Ru-Ru and LU Meng and, et al. Applications of Deep Learning Techniques for Diabetic Retinal Diagnosis. Acta Automatica Sinica, 2019
    [26] 陈文帅, 任志刚, 吴宗泽. 基于深度学习的极性电子元器件目标检测与方向识别方法. 自动化学报, 2019: 1−10 doi: 10.16383/j.aas.c190037

    Chen Wen-Shuai, Ren Zhi-Gang, Wu Zong-Ze, Fu Min-Yue. Detecting Object and Direction for Polar Electronic Components via Deep Learning. Acta Automatica Sinica, 2019: 1−10 doi: 10.16383/j.aas.c190037
    [27] 吴高昌, 刘强, 柴天佑, 秦泗钊. 基于时序图像深度学习的电熔镁炉异常工况诊断. 自动化学报, 2019, 45(8): 1475−1485

    Wu Gao-Chang, Liu Qiang, Chai Tian-You, Qin S. Joe. Abnormal condition diagnosis through deep learning of image sequences for fused magnesium furnaces. Acta Automatica Sinica, 2019, 45(8): 1475−1485
    [28] 张芳, 王萌, 肖志涛, 吴骏, 耿磊, 童军, 王雯. 基于全卷积神经网络与低秩稀疏分解的显著性检测. 自动化学报, 2019, 45(11): 2148−2158

    Zhang Fang, Wang Meng, Xiao Zhi-Tao, Wu Jun, Geng Lei, Tong Jun, Wang Wen. Saliency detection via full convolution neural network and low rank sparse decomposition. Acta Automatica Sinica, 2019, 45(11): 2148−2158
    [29] 李良福, 马卫飞, 李丽, 陆铖. 基于深度学习的桥梁裂缝检测算法研究. 自动化学报, 2019, 45(9): 1727−1742

    Li Liang-Fu, Ma Wei-Fei, Li Li, Lu Cheng. Research on detection algorithm for bridge cracks based on deep learning. Acta Automatica Sinica, 2019, 45(9): 1727−1742
    [30] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 2980−2988, 2017.
    [31] Jiahui Yu, Yuning Jiang, Zhangyang Wang, Zhimin Cao, and Thomas Huang. Unitbox: An advanced object detection network. In Proceedings of the 24th ACM International Conference on Multimedia, pages 516−520. ACM, 2016.
    [32] H İbrahim Çelik, L Canan Dülger, and Mehmet Topalbekiroğlu. Fabric defect detection using linear filtering and morphological operations. 2014.
    [33] Michael K Ng, Henry YT Ngan, Xiaoming Yuan, and Wenxing Zhang. Patterned fabric inspection and visualization by the method of image decomposition. IEEE Transactions on Automation Science and Engineering, 2014, 11(3): 943−947 doi: 10.1109/TASE.2014.2314240
    [34] Ruoxu Ren, Terence Hung, and Kay Chen Tan. A generic deep-learning-based approach for automated surface inspection. IEEE Transactions on Cybernetics, 2017, 48(3): 929−940
    [35] Hİ Çelik, LC Dülger, and M Topalbekiroğlu. Development of a machine vision system: real-time fabric defect detection and classification with neural networks. The Journal of The Textile Institute, 2014, 105(6): 575−585 doi: 10.1080/00405000.2013.827393
    [36] Lucia Bissi, Giuseppe Baruffa, Pisana Placidi, Elisa Ricci, Andrea Scorzoni, and Paolo Valigi. Automated defect detection in uniform and structured fabrics using gabor filters and pca. Journal of Visual Communication and Image Representation, 2013, 24(7): 838−845 doi: 10.1016/j.jvcir.2013.05.011
    [37] Yundong Li, Weigang Zhao, and Jiahao Pan. Deformable patterned fabric defect detection with fisher criterion-based deep learning. IEEE Transactions on Automation Science and Engineering, 2016, 14(2): 1256−1264
    [38] Jun-Feng Jing, Hao Ma, and Huan-Huan Zhang. Automatic fabric defect detection using a deep convolutional neural network. Coloration Technology, 2019, 135(3): 213−223 doi: 10.1111/cote.12394
    [39] Jagdish Lal Raheja, Bandla Ajay, and Ankit Chaudhary. Real time fabric defect detection system on an embedded dsp platform. Optik-International Journal for Light and Electron Optics, 2013, 124(21): 5280−5284 doi: 10.1016/j.ijleo.2013.03.038
    [40] Xian Tao, Dapeng Zhang, Wenzhi Ma, Xilong Liu, and De Xu. Automatic metallic surface defect detection and recognition with convolutional neural networks. Applied Sciences, 2018, 8(9): 1575 doi: 10.3390/app8091575
    [41] He Di, Xu Ke, Zhou Peng, and Zhou Dongdong. Surface defect classification of steels with a new semi-supervised learning method. Optics and Lasers in Engineering, 2019, 117: 40−48 doi: 10.1016/j.optlaseng.2019.01.011
    [42] Arikan, Selim and Varanasi, Kiran and Stricker, Didier. Surface Defect Classification in Real-Time Using Convolutional Neural Networks. arXiv preprint arXiv: 1904.04671, 2019.
  • 加载中
计量
  • 文章访问数:  64
  • HTML全文浏览量:  26
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-03-20
  • 录用日期:  2020-06-19

目录

    /

    返回文章
    返回