-
摘要: 针对热轧带钢表面缺陷检测存在的智能化水平低、检测精度低和检测速度慢等问题, 本文提出了一种基于自适应全局定位网络(Adaptive global localization network, AGLNet)的深度学习缺陷检测算法. 首先, 引入了一种残差网络(Residual network, ResNet)与特征金字塔网络(Feature pyramid network, FPN)集成的特征提取结构, 减少缺陷语义信息在层级传递间的消失; 其次, 提出基于Tree-structure parzen estimation的自适应树型候选框提取网络(Adaptive tree-structure region proposal network, AT-RPN), 无需先验知识的测试积累, 避免了人为调参的训练模; 最后, 引入了全局定位算法(Global localization regression)算法以全局定位的模式在复杂的缺陷检测中实现缺陷更精确定位.本文实现一种快速、准确、更智能化、更适用于实际工业应用的热轧带钢表面缺陷的算法.实验结果表明, AGLNet在NEU-DET热轧带钢表面缺陷数据集上的检测速度保持在11.8fps, 平均精度达到了79.90 %, 优于目前其他深度学习带钢表面缺陷检测算法; 另外该算法还具备较强的泛化能力.
-
关键词:
- 表面缺陷检测 /
- 深度学习 /
- 特征金字塔 /
- 自适应树形候选框提取 /
- 全局定位
Abstract: A deep learning defect detection model based on Adaptive global localization network (AGLNet) is presented to solve the problems of low intelligence, low detection accuracy and slow detection speed in Hot-rolled strip surface defect detection. First, the feature extraction structure is combined with Residual network (ResNet) and Feature pyramid network (FPN) to reduce the disappearance of defect semantic information between layers transfers. Secondly, an Adaptive tree-structure region proposal network (AT-RPN) based on Tree-structure parzen estimation algorithm is proposed, which does not need the accumulation of prior knowledge, and avoids the training model by manual parameter adjustment. Finally, a Global localization regression algorithm is proposed to locate defects more accurately in complex defect detection using global positioning mode. In this paper, a fast, accurate, more intelligent algorithm for surface defects detection of hot-rolled strips is realized. The experimental results show that the detection speed of AGLNet remains 11.8 fps and the average accuracy is 79.90 %, which is better than other deep learning algorithms for strip surface defect detection on NEU-DET dataset. In addition, the algorithm has a strong generalization ability. -
图 2 TPE自适应anchor-ratio调节模块流程图
Fig. 2 Flow chart of TPE adaptive anchor ratio adjustment module
图 4 AGLNet与Faster R-CNN和Grid R-CNN的比较
Fig. 4 Comparison of AGLnet with Fast R-CNN and Grid R-CNN
图 6 AT-RPN, RPN和AABO的分类损失函数变化对比
Fig. 6 The change of classification loss function of AT-RPN, RPN and AABO
图 7 AT-RPN, RPN和AABO的的位置回归损失函数变化对比
Fig. 7 The change of bounding box regression loss function of AT-RPN, RPN and AABO
图 8 PCB-Master数据集中的高宽比统计结果
Fig. 8 Statistical results of aspect ratio in PCB master dataset
图 10 AGLNet模型下裂纹和压入氧化缺陷检测结果与人工标注位置对
Fig. 10 Comparison between inspection results of Crazing and Rolled-in_scale defects under AGLNet model and manually marked positions
表 1 AGLNet、Gird R-CNN and Faster R-CNN基于NEU-DET数据集的对比测试结果
Table 1 Comparison results of AGLNet, Gird R-CNN and Fast R-CNN based on NEU-DET dataset
裂纹 夹杂 斑块 麻点 压入氧化 划痕 AGLNet 
Grid R-CNN 
Faster R-CNN 
下载: 导出CSV
表 2 各个模型在NEU-DET数据集的缺陷检测平均精度结果(%)
Table 2 Average accuracy results of defect detection on NEU-DET dataset of comparative experiment (%)
方法 平均精度均值 裂纹 夹杂 斑块 麻点 压入氧化 划痕 Faster R-CNN 79.20 71.31 84.63 82.92 80.17 80.31 75.87 RetinaNet 75.36 53.02 78.74 93.33 91.37 62.21 73.49 FCOS 75.18 52.41 75.03 91.48 84.85 62.86 84.43 Grid R-CNN 73.14 41.52 78.68 86.23 86.47 59.74 86.21 YOLO-v1 62.90 42.35 63.42 68.23 66.49 69.37 67.53 YOLO-v2 66.53 47.35 70.47 72.23 65.82 65.49 77.84 YOLO-v3 69.40 68.39 61.88 71.44 68.33 72.66 73.71 YOLO-v4 77.99 64.87 70.84 93.24 83.83 69.52 85.63 YOLO-v5 76.82 62.42 75.76 84.23 81.27 64.59 92.63 YOLOF 77.32 63.48 71.82 90.56 85.21 64.24 88.63 AGLNet 79.90 54.72 83.31 88.63 91.67 64.42 96.64
下载: 导出CSV
表 3 各模型FLOPs, Params和FPS对比结果
Table 3 Comparison of Flops, Params and FPS of each model
方法 FLOPs Params Fps Faster R-CNN 408.36GMac 98.25 M $\sim$8.2 RetinaNet 239.32GMac 37.74 M $\sim$12.3 FCOS 438.68GMac 89.79 M $\sim$9.3 Grid R-CNN 329.51GMac 64.32 M $\sim$10.2 YOLO-v3 89.45GMac 27.84 M $\sim$15.4 YOLOF 151.47GMac 63.24M $\sim$13.4 AGLNet 273.95GMac 79.8 M $\sim$11.8
下载: 导出CSV
表 4 各类缺陷在不同IoU阈值下的测试结果
Table 4 Detection results of various defects under different IOU thresholds
IoU阈值 缺陷类型 gts Dets Recall mAP IoU0.5 裂纹 139 1 886 0.935 54.72 IoU0.75 裂纹 139 1 823 0.923 47.48 IoU0.5 夹杂 181 1 188 0.945 83.31 IoU0.75 夹杂 181 1 163 0.932 82.17 IoU0.5 斑块 151 627 0.960 88.63 IoU0.75 斑块 151 591 0.942 89.45 IoU0.5 麻点 88 689 0.955 91.67 IoU0.75 麻点 88 636 0.938 89.24 IoU0.5 压入氧化 126 1 034 0.893 64.42 IoU0.75 压入氧化 126 1 051 0.882 59.66 IoU0.5 划痕 117 317 0.991 96.64 IoU0.75 划痕 117 322 0.986 92.79 IoU0.5 全部缺陷 802 5 741 0.947 79.90 IoU0.75 全部缺陷 802 5 586 0.934 76.79
下载: 导出CSV
表 5 消融实验结果
Table 5 Ablation results
序号 ResNet50+FPN ResNet50 AT-RPN RPN mAP（%） fps GPU Memory Usage 1 √ √ 79.90 11.8 5568MiB 2 √ √ 78.64 10.3 7039MiB 3 √ √ 77.97 12.2 5024MiB 4 √ √ 76.82 10.6 6436MiB
下载: 导出CSV
表 6 消融实验对比结果
Table 6 Comparison results of ablation experiments
序号 对比试验 mAP提升 fps提升 节约显存占用率 1 实验1*实验2 1.26% 1.5 20.89% 2 实验3*实验4 1.15% 1.6 21.93% 3 实验1*实验3 1.93% -0.4 -10.82% 4 实验2*实验4 1.82% -0.3 -9.36% 5 实验1*实验4 3.08% 1.2 13.49%
下载: 导出CSV
表 7 PCB-Master数据集基本信息
Table 7 Basic information of PCB master dataset
缺陷类型 图像数量 缺陷数量 漏孔(Missing_hole) 115 497 鼠咬(Mouse_bite) 115 492 断路(Open_circuit) 115 482 短路(Short) 115 491 毛刺(Spur) 115 488
下载: 导出CSV
表 8 各个模型在PCB-Master数据集上测试结果(%)
Table 8 Test results of each model on PCB master dataset (%)
Faster R-CNN RetinaNet FCOS Grid R-CNN Yolo-v3 YOLOF AGLNet MAP(%) 86.9 91.16 88.9 94.3 79.8 95.0 96.9 漏孔 0.874 0.915 0.907 0.956 0.858 0.942 0.995 鼠咬 0.849 0.905 0.852 0.934 0.793 0.934 0.952 断路 0.862 0.897 0.847 0.915 0.747 0.886 0.929 短路 0.895 0.922 0.928 0.997 0.832 0.997 0.997 毛刺 0.869 0.953 0.915 0.954 0.826 0.989 0.997 余铜 0.865 0.875 0.880 0.905 0.731 0.954 0.942
下载: 导出CSV
表 9 PCB-Master测试集检测数据统计
Table 9 Data statistics of PCB-Master defect detection test set
缺陷类别 gts Dets recall ap 漏孔(Missing_hole) 169 696 0.998 0.995 鼠咬(Mouse_bite) 142 665 0.990 0.952 断路(Open_circuit) 142 667 0.990 0.929 短路(Short) 132 590 1 0.997 毛刺(Spur) 143 687 1 0.997 余铜(Spurious_copper) 137 644 0.979 0.942 全部缺陷总计 865 3949
下载: 导出CSV
-
[1] 王典洪,甘胜丰,张伟民,雷维新. 基于监督双限制连接Isomap算法的带钢表面缺陷图像分类方法. 自动化学报, 2014, 40(5): 883-891Wang D H, Gan S F, Zhang W M, Lei W X. Strip Surface Defect Image Classification Based on Double-limited and Supervised-connect Isomap Algorithm. Acta Automatica Sinica, 2014, 40(5): 883-891 [2] Song, K C and Yan Y H. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science, 2013, 285(x): 858-864 [3] Neogi, N, Mohanta, D K, Dutta, P K. Review of vision-based steel surface inspection systems. EURASIP Journal on Image and Video Processing, 2014, 1-19 [4] 许志祥,卢宏,沈剑. 摄像机定标及其误差分析. 自动化学报, 1993, 01(x): 115-117 doi: 10.16383/j.aas.1993.01.018Xu Z X, Lu H, Shen J.Camera Calibration and its Error Analysis. Acta Automatica Sinica, 1993, 01(x): 115-117 doi: 10.16383/j.aas.1993.01.018 [5] 李少波,杨静,王铮,朱书德,杨观赐. 缺陷检测技术的发展与应用研究综述. 自动化学报, 2020, 46(11): 2319-2336 doi: 10.16383/j.aas.c180538Li S B, Yang J, Wang Z, Zhu S D, Yang G C. Review of development and application of defect detection technology. Acta Automatica Sinica, 2020, 46(11): 2319-2336 doi: 10.16383/j.aas.c180538 [6] 刘国梁,余建波. 基于堆叠降噪自编码器的神经-符号模型及在晶圆表面缺陷识别. 自动化学报, 2021, x(x): 1-15Liu G L, Yu J B. Application of neural-symbol model based on stacked denoising auto-encoders in wafer map defect recognition. Acta Automatica Sinica, 2021, x(x): 1-15 [7] Ren S, He K, Girshick R B, Sun J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(x): 1137-1149 [8] He K, Gkioxari G, Dollár P Girshick R B. Mask R-CNN. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(x): 386-397. [9] Redmon J, Divvala S, Girshick R B, Farhadi A. You Only Look Once: Unified, Real-Time Object Detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016: 779−788 [10] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C, Berg A. SSD: Single Shot MultiBox Detector. In: 2016 Proceedings of the European Conference on Computer Vision (ECCV). 2016 [11] Lin T, Dollár P, Girshick R B, He K, Hariharan B, Belongie S J. Feature Pyramid Networks for Object Detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017: 936−944 [12] Tao X, Zhang D, Wang Z, Liu X, Zhang H, Xu D. Detection of Power Line Insulator Defects Using Aerial Images Analyzed With Convolutional Neural Networks. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2020, 50(x): 1486-1498 [13] He Y, Song K, Meng Q, Yan Y. An End-to-End Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features. IEEE Transactions on Instrumentation and Measurement, 2020, 69(x): 1493-1504 [14] Cheng X, Yu J. RetinaNet With Difference Channel Attention and Adaptively Spatial Feature Fusion for Steel Surface Defect Detection. IEEE Transactions on Instrumentation and Measurement, 2021, 70(x): 1-11 [15] Chen J, Liu Z, Wang H, Núñez A, Han Z. Automatic Defect Detection of Fasteners on the Catenary Support Device Using Deep Convolutional Neural Network. IEEE Transactions on Instrumentation and Measurement, 2018, 67(x): 257-269 [16] Zhang C B, Chang C C, Jamshidi M. Concrete bridge surface damage detection using a single-stage detector. IComputer-Aided Civil and Infrastructure Engineering, 2020, 35(x): 389-409 [17] Zhang S, Chi C, Yao Y, Lei Z, Li S. Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020: 9756−9765 [18] Law H, Deng J. CornerNet: Detecting Objects as Paired Keypoints. International Journal of Computer Vision, 2019, 128(x): 642-656 [19] Duan K, Bai S, Xie L, Qi H, Huang Q, Tian Q. CenterNet: Keypoint Triplets for Object Detection. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2019: 6568−6577 [20] Jia X, Yang X, Yu X, Gao H. A Modified CenterNet for Crack Detection of Sanitary Ceramics. In: IECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society. 2020: 5311−5316 [21] Zhu C, He Y, Savvides M. Feature Selective Anchor-Free Module for Single-Shot Object Detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 840−849 [22] Tian Z, Shen C, Chen H, He T. FCOS: Fully Convolutional One-Stage Object Detection. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2019: 9626-9635. [23] Kong T, Sun F, Liu H, Jiang Y, Li L, Shi J. FoveaBox: Beyound Anchor-Based Object Detection. IEEE Transactions on Image Processing, 2019, 29(x): 7389-7398 [24] Lu X, Li B, Yue Y, Li Q, Yan J. Grid R-CNN. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 7355−7364 [25] Wang J, Zhang W, Cao Y, Chen, K, Pang J, Gong T, Shi J, Loy C C, Lin D. Side-Aware Boundary Localization for More Precise Object Detection. In: 2020 Proceedings of the European Conference on Computer Vision(ECCV). 2020 [26] He Y, Song K, Meng Q, Yan Y. An End-to-End Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features. IEEE Transactions on Instrumentation and Measurement, 2019, 69(x): 1493-1504 [27] Bergstra J, Bardenet R, Bengio Y, Kégl B. Algorithms for Hyper-Parameter Optimization. NIPS. 2011 [28] Cao J, Cholakkal H, Anwer R M, Khan F, Pang Y, Shao L. D2Det: Towards High Quality Object Detection and Instance Segmentation. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020: 11482−11491 [29] Ding R, Dai L, Li G, Liu H. TDD-net: a tiny defect detection network for printed circuit boards. CAAI Trans. Intell. Technol, 2019, 4(x): 110-116 -
点击查看大图
计量
- 文章访问数: 470
- HTML全文浏览量: 183
- 被引次数: 0
