2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于自适应全局定位算法的带钢表面缺陷检测

王延舒 余建波

王延舒, 余建波. 基于自适应全局定位算法的带钢表面缺陷检测. 自动化学报, 2024, 50(8): 1−15 doi: 10.16383/j.aas.c210467
引用本文: 王延舒, 余建波. 基于自适应全局定位算法的带钢表面缺陷检测. 自动化学报, 2024, 50(8): 1−15 doi: 10.16383/j.aas.c210467
Wang Yan-Shu, Yu Jian-Bo. Strip surface defect detection based on adaptive global localization algorithm. Acta Automatica Sinica, 2024, 50(8): 1−15 doi: 10.16383/j.aas.c210467
Citation: Wang Yan-Shu, Yu Jian-Bo. Strip surface defect detection based on adaptive global localization algorithm. Acta Automatica Sinica, 2024, 50(8): 1−15 doi: 10.16383/j.aas.c210467

基于自适应全局定位算法的带钢表面缺陷检测

doi: 10.16383/j.aas.c210467
基金项目: 国家自然科学基金(71777173), 上海科委“科技创新行动计划”高新技术领域项目(19511106303), 中央高校基本业务经费项目资助
详细信息
    作者简介:

    王延舒:同济大学机械与能源工程学院硕士研究生. 2020年获四川大学学士学位. 主要研究方向为机器学习, 深度学习, 视觉检测与识别. E-mail: 2030211@tongji.edu.cn

    余建波:同济大学机械与能源工程学院教授. 2009年获上海交通大学机械工程学院博士学位. 主要研究方向为机器学习, 深度学习, 智能质量管控, 过程控制, 视觉检测与识别. 本文通信作者. E-mail: jbyu@tongji.edu.cn

Strip Surface Defect Detection Based on Adaptive Global Localization Algorithm

Funds: Supported by National Natural Science Foundation of China (71777173), “Action Plan for Scientific and Technological Innovation” of Shanghai Science and Technology Commission (19511106303), Fundamental Research Funds for the Central Universities
More Information
    Author Bio:

    WANG Yan-Shu Master student at the Mechanical and Energy Engineering, Tongji University. He received his bachelor degree from Sichuan University. His research interest covers machine learning deep learing, and visual detection and recognition

    YU Jian-Bo Professor at the Mechanical and Energy Engineering, Tongji University. He received his Ph.D. degree from Shanghai Jiao Tong University. His research interest covers machine learning, deep learning, intelligent quality control, process control, visual inspection and identification. Corresponding author of this paper

  • 摘要: 针对热轧带钢表面缺陷检测存在的智能化水平低、检测精度低和检测速度慢等问题, 提出了一种基于自适应全局定位网络(Adaptive global localization network, AGLNet)的深度学习缺陷检测算法. 首先, 引入一种残差网络(Residual network, ResNet)与特征金字塔网络(Feature pyramid network, FPN)集成的特征提取结构, 减少缺陷语义信息在层级传递间的消失; 其次, 提出基于TPE (Tree-structure parzen estimation)的自适应树型候选框提取网络(Adaptive tree-structure region proposal extraction network, AT-RPN), 无需先验知识的积累, 避免了人为调参的训练模式; 最后, 引入全局定位回归(Global localization regression)算法以全局定位的模式在复杂的缺陷检测中实现缺陷更精确定位. 本文实现一种快速、准确、更智能化、更适用于实际应用的热轧带钢表面缺陷的算法. 实验结果表明, AGLNet在NEU-DET热轧带钢表面缺陷数据集上的检测速度保持在11.8帧/s, 平均精度达到79.90 %, 优于目前其他深度学习带钢表面缺陷检测算法. 另外, 该算法还具备较强的泛化能力.
  • 图  1  AGLNet网络

    Fig.  1  The structure of AGLNet

    图  2  TPE自适应Anchor-ratio调节模块流程图

    Fig.  2  Flow chart of TPE adaptive anchor-ratio adjustment module

    图  3  AT-RPN整体结构图

    Fig.  3  Whole structure of AT-RPN

    图  4  AGLNet与Faster R-CNN和Grid R-CNN的比较

    Fig.  4  Comparison of AGLNet with Fast R-CNN and Grid R-CNN

    图  5  NEU-DET数据集热轧带钢表面缺陷

    Fig.  5  Surface defects of hot rolled strip in NET-DET dataset

    图  6  AT-RPN, RPN和AABO的分类损失函数变化对比

    Fig.  6  The change of classification loss function of AT-RPN, RPN and AABO

    图  7  AT-RPN, RPN和AABO的的位置回归损失函数变化对比

    Fig.  7  The change of bounding box regression loss function of AT-RPN, RPN and AABO

    图  8  PCB-Master数据集中的高宽比统计结果

    Fig.  8  Statistical results of aspect ratio in PCB-Master dataset

    图  9  PCB-Master检测结果

    Fig.  9  PCB-Master test results

    图  10  AGLNet模型下裂纹和压入氧化缺陷检测结果与人工标注位置对

    Fig.  10  Comparison between inspection results of Crazing and rolled-in_scale defects under AGLNet model and manually marked positions

    表  1  AGLNet、Gird R-CNN and Faster R-CNN基于NEU-DET数据集的对比测试结果

    Table  1  Comparison results of AGLNet, Gird R-CNN and Fast R-CNN based on NEU-DET dataset

    裂纹 夹杂 斑块 麻点 压入氧化 划痕
    AGLNet
    Grid R-CNN
    Faster R-CNN
    下载: 导出CSV

    表  2  各个模型在NEU-DET数据集的缺陷检测平均精度结果(%)

    Table  2  Average accuracy results of defect detection on NEU-DET dataset of comparative experiment (%)

    方法 平均精度均值 裂纹 夹杂 斑块 麻点 压入氧化 划痕
    Faster R-CNN 79.20 71.31 84.63 82.92 80.17 80.31 75.87
    RetinaNet 75.36 53.02 78.74 93.33 91.37 62.21 73.49
    FCOS 75.18 52.41 75.03 91.48 84.85 62.86 84.43
    Grid R-CNN 73.14 41.52 78.68 86.23 86.47 59.74 86.21
    YOLO-v1 62.90 42.35 63.42 68.23 66.49 69.37 67.53
    YOLO-v2 66.53 47.35 70.47 72.23 65.82 65.49 77.84
    YOLO-v3 69.40 68.39 61.88 71.44 68.33 72.66 73.71
    YOLO-v4 77.99 64.87 70.84 93.24 83.83 69.52 85.63
    YOLO-v5 76.82 62.42 75.76 84.23 81.27 64.59 92.63
    YOLOF 77.32 63.48 71.82 90.56 85.21 64.24 88.63
    AGLNet 79.90 54.72 83.31 88.63 91.67 64.42 96.64
    下载: 导出CSV

    表  3  各模型FLOPs, Params和FPS对比结果

    Table  3  Comparison of FLOPs, Params and FPS of each model

    方法 FLOPs (GMAC) Params (M) FPS (帧/s)
    Faster R-CNN 408.36G 98.25 $\sim$8.2
    RetinaNet 239.32G 37.74 $\sim$12.3
    FCOS 438.68G 89.79 $\sim$9.3
    Grid R-CNN 329.51G 64.32 $\sim$10.2
    YOLO-v3 89.45G 27.84 $\sim$15.4
    YOLOF 151.47G 63.24 $\sim$13.4
    AGLNet 273.95G 79.8 $\sim$11.8
    下载: 导出CSV

    表  4  各类缺陷在不同IoU阈值下的测试结果

    Table  4  Detection results of various defects under different IoU thresholds

    IoU阈值 缺陷类型 gts Dets Recall mAP
    IoU0.5 裂纹 139 1 886 0.935 54.72
    IoU0.75 裂纹 139 1 823 0.923 47.48
    IoU0.5 夹杂 181 1 188 0.945 83.31
    IoU0.75 夹杂 181 1 163 0.932 82.17
    IoU0.5 斑块 151 627 0.960 88.63
    IoU0.75 斑块 151 591 0.942 89.45
    IoU0.5 麻点 88 689 0.955 91.67
    IoU0.75 麻点 88 636 0.938 89.24
    IoU0.5 压入氧化 126 1 034 0.893 64.42
    IoU0.75 压入氧化 126 1 051 0.882 59.66
    IoU0.5 划痕 117 317 0.991 96.64
    IoU0.75 划痕 117 322 0.986 92.79
    IoU0.5 全部缺陷 802 5 741 0.947 79.90
    IoU0.75 全部缺陷 802 5 586 0.934 76.79
    下载: 导出CSV

    表  5  消融实验结果

    Table  5  Results of ablation experiments

    序号 ResNet50 + FPN ResNet50 AT-RPN RPN mAP(%) FPS GPU 存贮占用量(MiB)
    1 79.90 11.8 5568
    2 78.64 10.3 7039
    3 77.97 12.2 5024
    4 76.82 10.6 6436
    下载: 导出CSV

    表  6  消融实验对比结果

    Table  6  Comparison results of ablation experiments

    序号 对比实验 mAP提升(%) FPS提升 节约显存占用率(%)
    1 实验1/实验2 1.26 1.5 20.89
    2 实验3/实验4 1.15 1.6 21.93
    3 实验1/实验3 1.93 −0.4 −10.82
    4 实验2/实验4 1.82 −0.3 −9.36
    5 实验1/实验4 3.08 1.2 13.49
    下载: 导出CSV

    表  7  PCB-Master数据集基本信息

    Table  7  Basic information of PCB master dataset

    缺陷类型 图像数量 缺陷数量
    漏孔 115 497
    鼠咬 115 492
    断路 115 482
    短路 115 491
    毛刺 115 488
    下载: 导出CSV

    表  8  各个模型在PCB-Master数据集上测试结果(%)

    Table  8  Test results of each model on PCB master dataset (%)

    Faster R-CNN RetinaNet FCOS Grid R-CNN Yolo-v3 YOLOF AGLNet
    mAP(%) 86.900 91.160 88.900 94.300 79.800 95.000 96.900
    漏孔 0.874 0.915 0.907 0.956 0.858 0.942 0.995
    鼠咬 0.849 0.905 0.852 0.934 0.793 0.934 0.952
    断路 0.862 0.897 0.847 0.915 0.747 0.886 0.929
    短路 0.895 0.922 0.928 0.997 0.832 0.997 0.997
    毛刺 0.869 0.953 0.915 0.954 0.826 0.989 0.997
    余铜 0.865 0.875 0.880 0.905 0.731 0.954 0.942
    下载: 导出CSV

    表  9  PCB-Master测试集检测数据统计

    Table  9  Data statistics of PCB-Master defect detection test set

    缺陷类别 gts Dets Recall AP
    漏孔 169 696 0.998 0.995
    鼠咬 142 665 0.990 0.952
    断路 142 667 0.990 0.929
    短路 132 590 1 0.997
    毛刺 143 687 1 0.997
    余铜 137 644 0.979 0.942
    全部缺陷总计 865 3949
    下载: 导出CSV
  • [1] 王典洪, 甘胜丰, 张伟民, 雷维新. 基于监督双限制连接Isomap算法的带钢表面缺陷图像分类方法. 自动化学报, 2014, 40(5): 883−891

    Wang Dian-Hong, Gan Sheng-Feng, Zhang Wei-Min, Lei Wei-Xin. Strip surface defect image classification based on double-limited and supervised-connect Isomap algorithm. Acta Automatica Sinica, 2014, 40(5): 883−891
    [2] Song K C, Yan Y H. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science, 2013, 285: 858−864 doi: 10.1016/j.apsusc.2013.09.002
    [3] Neogi N, Mohanta D K, Dutta P K. Review of vision-based steel surface inspection systems. EURASIP Journal on Image and Video Processing, 2014, 2014(1): Article No. 50 doi: 10.1186/1687-5281-2014-50
    [4] 许志祥, 卢宏, 沈剑. 摄像机定标及其误差分析. 自动化学报, 1993, 19(1): 115−117

    Xu Zhi-Xiang, Lu Hong, Shen Jian. Camera calibration and its error analysis. Acta Automatica Sinica, 1993, 19(1): 115−117
    [5] 李少波, 杨静, 王铮, 朱书德, 杨观赐. 缺陷检测技术的发展与应用研究综述. 自动化学报, 2020, 46(11): 2319−2336

    Li Shao-Bo, Yang Jing, Wang Zheng, Zhu Shu-De, Yang Guan-Ci. Review of development and application of defect detection technology. Acta Automatica Sinica, 2020, 46(11): 2319−2336
    [6] 刘国梁, 余建波. 基于堆叠降噪自编码器的神经-符号模型及在晶圆表面缺陷识别. 自动化学报, 2022, 48(11): 2688−2702

    Liu Guo-Liang, Yu Jian-Bo. Application of neural-symbol model based on stacked denoising auto-encoders in wafer map defect recognition. Acta Automatica Sinica, 2022, 48(11): 2688−2702
    [7] Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137−1149 doi: 10.1109/TPAMI.2016.2577031
    [8] He K M, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 386−397 doi: 10.1109/TPAMI.2018.2844175
    [9] Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016. 779−788
    [10] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y, et al. SSD: Single shot MultiBox detector. In: Proceedings of the 14th European Conference on Computer Vision (ECCV). Amsterdam, The Netherlands: Springer, 2016. 21−37
    [11] Lin T Y, Dollár P, Girshick R, He K M, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 936−944
    [12] Tao X, Zhang D P, Wang Z H, Liu X L, Zhang H Y, Xu D. Detection of power line insulator defects using aerial images analyzed with convolutional neural networks. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2020, 50(4): 1486−1498 doi: 10.1109/TSMC.2018.2871750
    [13] He Y, Song K C, Meng Q G, Yan Y H. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Transactions on Instrumentation and Measurement, 2020, 69(4): 1493−1504 doi: 10.1109/TIM.2019.2915404
    [14] Cheng X, Yu J B. RetinaNet with difference channel attention and adaptively spatial feature fusion for steel surface defect detection. IEEE Transactions on Instrumentation and Measurement, 2021, 70: Article No. 2503911
    [15] Chen J W, Liu Z G, Wang H R, Núñez A, Han Z W. Automatic defect detection of fasteners on the catenary support device using deep convolutional neural network. IEEE Transactions on Instrumentation and Measurement, 2018, 67(2): 257−269 doi: 10.1109/TIM.2017.2775345
    [16] Zhang C B, Chang C C, Jamshidi M. Concrete bridge surface damage detection using a single-stage detector. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(4): 389−409 doi: 10.1111/mice.12500
    [17] Zhang S F, Chi C, Yao Y Q, Lei Z, Li S Z. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 9756−9765
    [18] Law H, Deng J. CornerNet: Detecting objects as paired keypoints. International Journal of Computer Vision, 2020, 128(3): 642−656 doi: 10.1007/s11263-019-01204-1
    [19] Duan K W, Bai S, Xie L X, Qi H G, Huang Q M, Tian Q. CenterNet: Keypoint triplets for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South): IEEE, 2019. 6568−6577
    [20] Jia X G, Yang X Q, Yu X H, Gao H J. A modified CenterNet for crack detection of sanitary ceramics. In: Proceedings of the 46th Annual Conference of the IEEE Industrial Electronics Society (IECON). Singapore: IEEE, 2020. 5311−5316
    [21] Zhu C C, He Y H, Savvides M. Feature selective anchor-free module for single-shot object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA: IEEE, 2019. 840−849
    [22] Tian Z, Shen C H, Chen H, He T. FCOS: Fully convolutional one-stage object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South): IEEE, 2019. 9626−9635.
    [23] Kong T, Sun F C, Liu H P, Jiang Y N, Li L, Shi J B. FoveaBox: Beyound anchor-based object detection. IEEE Transactions on Image Processing, 2020, 29: 7389−7398 doi: 10.1109/TIP.2020.3002345
    [24] Lu X, Li B Y, Yue Y X, Li Q Q, Yan J J. Grid R-CNN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA: IEEE, 2019. 7355−7364
    [25] Wang J Q, Zhang W W, Cao Y H, Chen K, Pang J M, Gong T, et al. Side-aware boundary localization for more precise object detection. In: Proceedings of the 16th European Conference on Computer Vision (ECCV). Glasgow, UK: Springer, 2020. 403−419
    [26] He Y, Song K C, Meng Q G, Yan Y H. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Transactions on Instrumentation and Measurement, 2020, 69(4): 1493−1504>查阅网上资料, 本条文献与第13条文献重复, 请确认) doi: 10.1109/TIM.2019.2915404

    He Y, Song K C, Meng Q G, Yan Y H. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Transactions on Instrumentation and Measurement, 2020, 69(4): 1493-1504查阅网上资料, 本条文献与第13条文献重复, 请确认) doi: 10.1109/TIM.2019.2915404
    [27] Bergstra J, Bardenet R, Bengio Y, Kégl B. Algorithms for hyper-parameter optimization. In: Proceedings of the 24th International Conference on Neural Information Processing Systems (NIPS). Granada, Spain: Curran Associates Inc., 2011. 2546−2554
    [28] Cao J L, Cholakkal H, Anwer R M, Khan F S, Pang Y W, Shao L. D2Det: Towards high quality object detection and instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 11482−11491
    [29] Ding R W, Dai L H, Li G P, Liu H. TDD-net: A tiny defect detection network for printed circuit boards. CAAI Transactions on Intelligence Technology, 2019, 4(2): 110−116 doi: 10.1049/trit.2019.0019
  • 加载中
计量
  • 文章访问数:  538
  • HTML全文浏览量:  222
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-07-23
  • 录用日期:  2021-11-26
  • 网络出版日期:  2023-02-06

目录

    /

    返回文章
    返回