2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于计算机视觉的工业金属表面缺陷检测综述

伍麟 郝鸿宇 宋友

伍麟, 郝鸿宇, 宋友. 基于计算机视觉的工业金属表面缺陷检测综述. 自动化学报, 2024, 50(7): 1261−1283 doi: 10.16383/j.aas.c230039
引用本文: 伍麟, 郝鸿宇, 宋友. 基于计算机视觉的工业金属表面缺陷检测综述. 自动化学报, 2024, 50(7): 1261−1283 doi: 10.16383/j.aas.c230039
Wu Lin, Hao Hong-Yu, Song You. A review of metal surface defect detection based on computer vision. Acta Automatica Sinica, 2024, 50(7): 1261−1283 doi: 10.16383/j.aas.c230039
Citation: Wu Lin, Hao Hong-Yu, Song You. A review of metal surface defect detection based on computer vision. Acta Automatica Sinica, 2024, 50(7): 1261−1283 doi: 10.16383/j.aas.c230039

基于计算机视觉的工业金属表面缺陷检测综述

doi: 10.16383/j.aas.c230039
基金项目: 河北省重点研发计划 (21310101D)资助
详细信息
    作者简介:

    伍麟:北京航空航天大学硕士研究生. 主要研究方向为计算机视觉, 目标检测和表面缺陷检测. E-mail: zf2021349@buaa.edu.cn

    郝鸿宇:北京航空航天大学硕士研究生. 主要研究方向为计算机视觉, 图神经网络和少样本学习. E-mail: JoeyHao@buaa.edu.cn

    宋友:北京航空航天大学教授. 主要研究方向为软件工程, 异常信号检测, 算法分析与设计. 分别于1997年和2003年获得北京航空航天大学学士学位和博士学位. 本文通信作者. E-mail: songyou@buaa.edu.cn

A Review of Metal Surface Defect Detection Based on Computer Vision

Funds: Supported by Key Research & Development Plan of Hebei Province (21310101D)
More Information
    Author Bio:

    WU Lin Master student at the School of Software, Beihang University. His research interest covers computer vision, object detection and surface defect detection

    HAO Hong-Yu Master student at the School of Software, Beihang University. His research interest covers computer vision, graph neural network and few-shot learning

    SONG You Professor at the School of Software, Beihang University. He received his bachelor and Ph.D. degrees from Beihang University in 1997 and 2003, respectively. His research interest covers software engineering, anomaly signal detection, algorithm analysis and design. Corresponding author of this paper

  • 摘要: 针对平面及三维结构金属材料的工业表面缺陷检测, 概述了视觉检测技术的基本原理和研究现状, 并总结出视觉自动检测系统的关键技术包括光学成像技术、图像预处理技术与缺陷检测器. 首先介绍了如何根据检测对象的光学特性选择合适的二维、三维光学成像技术; 其次介绍了图像降噪、特征提取、图像分割和拼接等预处理技术的重要作用; 然后根据缺陷检测器的实现原理将其分为模板匹配、图像分类、图像语义分割、目标检测和图像异常检测五类, 并对其中的经典算法进行了归纳分析. 最后, 探讨了工业场景下金属表面缺陷检测技术实施中的关键问题, 并对该技术的发展趋势进行了展望.
  • 图  1  金属表面缺陷检测基本流程

    Fig.  1  Pipline of metal surface defect detection

    图  2  自动光学成像系统

    Fig.  2  Automated optical inspection system

    图  3  表面散射模型

    Fig.  3  Light scattering model on surface

    图  4  照明光路类型

    Fig.  4  Types of lighting path

    图  5  二维成像与三维成像对比

    Fig.  5  2D imaging versus 3D imaging

    图  6  光度立体法

    Fig.  6  Photometric stereo

    图  7  结构光法

    Fig.  7  Structured light illumination

    图  8  混合成像技术. ((a) ~ (c) 二维灰度图像 (d) ~ (f)叠加三维深度信息的图像)

    Fig.  8  Hybrid imaging technique. ((a) ~ (c) 2D grayscale images (d) ~ (f) Images with 3D depthinformation represented)

    图  9  基于图像分割的缺陷检测

    Fig.  9  Defect detection based on image segmentation

    图  10  三元网络结构

    Fig.  10  The structure of triplet network

    图  11  二阶段网络和一阶段网络对比

    Fig.  11  Comparison of two-stage and one-stage networks

    图  12  基于二阶段网络的金属表面缺陷检测,经许可转载自文献[100]

    Fig.  12  Metal surface defect detection based on two-stage networks, reproduced with permission from reference [100]

    图  13  DETR网络结构

    Fig.  13  The Network Architecture of DETR

    图  14  基于图像重建的缺陷检测. ((a)变分自编码机; (b) GAN结合AE; (c)基于记忆池的模型)

    Fig.  14  Defect detection based on image reconstruction. ((a) VAE; (b) GAN associated with Auto-Encoder; (c) Memory based model)

    图  15  基于标准化流的缺陷检测. ((a)原始图像; (b)多尺度输入; (c)图像特征分布; (d)简单分布; (e)标准分布; (f)异常分布)

    Fig.  15  Defect detection based on normalizing flow. ((a) Origin image; (b) Multiscale input; (c) Feature distribution; (d) Simple distribution; (e) Normalized distribution; (f) Anomalous distribution)

    图  16  (a)基于教师−学生网络的方法; (b)基于最典型嵌入表示的方法

    Fig.  16  (a) Method based on teacher-student network; (b) Method based on the most typical embedding representation

    表  1  目标检测算法在NEU-DET上的表现

    Table  1  Performance of object detection models on NEU-DET

    MethodBackboneNeck$AP_{50}$
    Faster R-CNN[97]ResNet-50FPN74.7
    Cascade R-CNN[99]ResNet-50FPN75.8
    YOLOX[115]CSPDarknetPA-FPN70.9
    YOLOv4[105]CSPDarknetFPN76.4
    AutoAssign[116]ResNet-50FPN76.6
    AutoAssign[116]Swin-TinyFPN78.3
    DDN[108]ResNet-50MFN82.3
    CA-AutoAssign[117]CSPDarknetCA82.7
    下载: 导出CSV

    表  2  异常检测方法对比

    Table  2  Comparison of abnormal detection

    MethodDetection AUROCSegmentation AUROCFPS
    PatchCore Large[145]99.698.25.9
    PNI[146]99.599.0
    MemSeg[144]99.599.631.3
    Fastflow[141]99.498.521.8
    EfficientAD-M[148]99.196.9269.0
    EfficientAD-S[148]98.896.8614.0
    CS-Flow[140]98.7
    Patch SVDD[142]92.195.72.1
    VAE-Grad[136]89.2
    下载: 导出CSV

    表  3  缺陷检测方法对比

    Table  3  Comparison of defect detection methods

    方法基本原理应用场景优缺点
    模板匹配比较模板与待检样本的差异来判断是否存在缺陷产品高度一致的金属精密加工制成品, 例如手机外壳、汽车零件等方法简单有效, 但需要提取制作模板, 仅适用于一致性强的产品
    分类网络直接用CNN提取特征, 通过Softmax或距离度量来预测类别公差较大、尺寸较小的金属制品, 例如螺母、金属盖等零件结构简单, 是其他网络的基础, 准确率依赖缺陷样本数量, 难以定位缺陷位置
    目标检测对每个提议候选框或者每个网格进行密集预测, 从背景中找出所有目标的分类和位置适用于绝大多数缺陷类别可事先定义的工业场景速度快, 适用范围广, 但网络结构复杂, 依赖大量缺陷样本进行训练
    语义分割通过卷积提取高阶语义特征, 然后通过上采样输出像素级的缺陷边界划分大面积金属板、带制品, 缺陷具有成片连续区域、形态不定的场景可以进行像素级缺陷分割, 但是依赖大量像素级标注数据, 标注成本很高
    异常检测通过自编码机、GAN、标准流等生成模型学习正常样本的表达方式, 根据重建误差、梯度或分布差异来进行缺陷检测缺乏缺陷样本, 只有正常样本可以用于训练的场景无需缺陷样本和标注, 可以检测未事先定义的缺陷类别, 但准确率尚达不到有监督学习的效果
    下载: 导出CSV
  • [1] Neogi N, Mohanta D K, Dutta P K. Review of vision-based steel surface inspection systems. EURASIP Journal on Image and Video Processing, 2014, 2014(1): Article No. 50 doi: 10.1186/1687-5281-2014-50
    [2] 卢荣胜, 吴昂, 张腾达, 王永红. 自动光学(视觉)检测技术及其在缺陷检测中的应用综述. 光学学报, 2018, 38(8): Article No. 0815002

    Lu Rong-Sheng, Wu Ang, Zhang Teng-Da, Wang Yong-Hong. Review on automated optical (visual) inspection and its applications in defect detection. Acta Optica Sinica, 2018, 38(8): Article No. 0815002
    [3] 吕承侃, 沈飞, 张正涛, 张峰. 图像异常检测研究现状综述. 自动化学报, 2022, 48(6): 1402−1428

    Lv Cheng-Kan, Shen Fei, Zhang Zheng-Tao, Zhang Feng. Review of image anomaly detection. Acta Automatica Sinica, 2022, 48(6): 1402−1428
    [4] 李维创, 尹柏强. 工业金属板带材表面缺陷自动视觉检测研究进展. 电子测量与仪器学报, 2021, 35(6): 1−16 doi: 10.13382/j.jemi.B2003349

    Li Wei-Chuang, Yin Bai-Qiang. Research progress of automated visual surface defect detection for industrial metal planar materials. Journal of Electronic Measurement and Instrumentation, 2021, 35(6): 1−16 doi: 10.13382/j.jemi.B2003349
    [5] 陶显, 侯伟, 徐德. 基于深度学习的表面缺陷检测方法综述. 自动化学报, 2021, 47(5): 1017−1034

    Tao Xian, Hou Wei, Xu De. A survey of surface defect detection methods based on deep learning. Acta Automatica Sinica, 2021, 47(5): 1017−1034
    [6] 罗东亮, 蔡雨萱, 杨子豪, 章哲彦, 周瑜, 白翔. 工业缺陷检测深度学习方法综述. 中国科学: 信息科学, 2022, 52(6): 1002−1039 doi: 10.1360/SSI-2021-0336

    Luo Dong-Liang, Cai Yu-Xuan, Yang Zi-Hao, Zhang Zhe-Yan, Zhou Yu, Bai Xiang. Survey on industrial defect detection with deep learning. Scientia Sinica Informationis, 2022, 52(6): 1002−1039 doi: 10.1360/SSI-2021-0336
    [7] Khan S, Naseer M, Hayat M, Zamir S W, Khan F S, Shah M. Transformers in vision: A survey. ACM Computing Surveys, 2022, 54(10s): Article No. 200
    [8] Huang Y C, Hung K C, Liu C C, Chuang T H, Chiou S J. Customized convolutional neural networks technology for machined product inspection. Applied Sciences, 2022, 12(6): Article No. 3014 doi: 10.3390/app12063014
    [9] Liu S, Wang Q, Luo Y. A review of applications of visual inspection technology based on image processing in the railway industry. Transportation Safety and Environment, 2019, 1(3): 185−204
    [10] Lu R S, Forrest A K. 3D surface topography from the specular lobe of scattered light. Optics and Lasers in Engineering, 2007, 45(10): 1018−1027 doi: 10.1016/j.optlaseng.2007.04.008
    [11] Smith B. Geometrical shadowing of a random rough surface. IEEE Transactions on Antennas and Propagation, 1967, 15(5): 668−671 doi: 10.1109/TAP.1967.1138991
    [12] Garcia-Lamont F, Cervantes J, López A, Rodriguez L. Segmentation of images by color features: A survey. Neurocomputing, 2018, 292: 1−27 doi: 10.1016/j.neucom.2018.01.091
    [13] Foucher B. Infrared machine vision: A new contender. In: Proceedings of the SPIE 3700, Thermosense XXI. Orlando, USA: SPIE, 1999. 210−213
    [14] Nayar S K, Ikeuchi K, Kanade T. Surface reflection: Physical and geometrical perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1991, 13(7): 611−634 doi: 10.1109/34.85654
    [15] Porter T F, Sylvester R A, Bouyoucas T W, Kolesar M P. Automatic strip surface defect detection system. Iron and Steel Engineer, 1988, 65(12): 17−20
    [16] Tian G Y, Lu R S, Gledhill D. Surface measurement using active vision and light scattering. Optics and Lasers in Engineering, 2007, 45(1): 131−139 doi: 10.1016/j.optlaseng.2006.03.005
    [17] 卢荣胜. 自动光学检测技术的发展现状. 红外与激光工程, 2008, 37(S1): 120−123

    Lu Rong-Sheng. State of the art of automated optical inspection. Infrared and Laser Engineering, 2008, 37(S1): 120−123
    [18] Rinn R, Thompson S A, Foehr R, Luecking F, Torre J. Parsytec HTS-2: Defect detection and classification through software vs. dedicated hardware. In: Proceedings of the Electronic Imaging '99. San Jose, CA, United States: SPIE, 1999. 110−121
    [19] 李松, 周亚同, 张忠伟, 池越, 韩春颖. 基于双打光模板匹配的冲压件表面缺陷检测. 锻压技术, 2018, 43(11): 137−145

    Li Song, Zhou Ya-Tong, Zhang Zhong-Wei, Chi Yue, Han Chun-Ying. Surface defect detection of stamping parts based on double light pattern matching. Forging & Stamping Technology, 2018, 43(11): 137−145
    [20] Landstrom A, Thurley M J. Morphology-based crack detection for steel slabs. IEEE Journal of Selected Topics in Signal Processing, 2012, 6(7): 866−875 doi: 10.1109/JSTSP.2012.2212416
    [21] Woodham R J. Photometric method for determining surface orientation from multiple images. Optical Engineering, 1980, 19(1): Article No. 191139
    [22] Berthold K P H. Shape From Shading: A Method for Obtaining the Shape of a Smooth Opaque Object from One View, Technical Report AITR-232, Artificial Intelligence Laboratory, MIT, USA, 1970.
    [23] Wang L, Xu K, Zhou P. Online detection technique of 3D defects for steel strips based on photometric stereo. In: Proceedings of the 8th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA). Macao, China: IEEE, 2016. 428−432
    [24] Delpy D T, Cope M, van der Zee P, Arridge S, Wray S, Wyatt J. Estimation of optical pathlength through tissue from direct time of flight measurement. Physics in Medicine & Biology, 1988, 33(12): 1433−1442
    [25] Bologna F, Tannous M, Romano D, Stefanini C. Automatic welding imperfections detection in a smart factory via 2-D laser scanner. Journal of Manufacturing Processes, 2022, 73: 948−960 doi: 10.1016/j.jmapro.2021.10.046
    [26] Huang C, Wang G L, Song H, Li R S, Zhang H O. Rapid surface defects detection in wire and arc additive manufacturing based on laser profilometer. Measurement, 2022, 189: Article No. 110503 doi: 10.1016/j.measurement.2021.110503
    [27] Li J L, Liu T, Wang X F. Advanced pavement distress recognition and 3D reconstruction by using GA-DenseNet and binocular stereo vision. Measurement, 2022, 201: Article No. 111760 doi: 10.1016/j.measurement.2022.111760
    [28] Li B Z, Xu Z J, Gao F, Cao Y L, Dong Q C. 3D reconstruction of high reflective welding surface based on binocular structured light stereo vision. Machines, 2022, 10(2): Article No. 159 doi: 10.3390/machines10020159
    [29] Zhou P, Xu K, Wang D D. Rail profile measurement based on line-structured light vision. IEEE Access, 2018, 6: 16423−16431 doi: 10.1109/ACCESS.2018.2813319
    [30] Guillo L, Jiang X R, Lafruit G, Guillemot C. Light Field Video Dataset Captured by A R8 Raytrix Camera (With Disparity Maps), Technical Report hal-01804578, International Organisation for Standardisation, San Diego, USA, 2018.
    [31] Saiz F A, Barandiaran I, Arbelaiz A, Graña M. Photometric stereo-based defect detection system for steel components manufacturing using a deep segmentation network. Sensors, 2022, 22(3): Article No. 882 doi: 10.3390/s22030882
    [32] Wen X, Song K C, Huang L M, Niu M H, Yan Y H. Complex surface ROI detection for steel plate fusing the gray image and 3D depth information. Optik, 2019, 198: Article No. 163313 doi: 10.1016/j.ijleo.2019.163313
    [33] Niu M H, Song K C, Huang L M, Wang Q, Yan Y H, Meng Q G. Unsupervised saliency detection of rail surface defects using stereoscopic images. IEEE Transactions on Industrial Informatics, 2021, 17(3): 2271−2281
    [34] Ren Z H, Fang F Z, Yan N, Wu Y. State of the art in defect detection based on machine vision. International Journal of Precision Engineering and Manufacturing-Green Technology, 2022, 9(2): 661−691 doi: 10.1007/s40684-021-00343-6
    [35] Niblack W. An Introduction to Digital Image Processing. Birkeroed: Strandberg Publishing Company, 1985.
    [36] Luisier F, Blu T, Unser M. A new SURE approach to image denoising: Interscale orthonormal wavelet thresholding. IEEE Transactions on Image Processing, 2007, 16(3): 593−606 doi: 10.1109/TIP.2007.891064
    [37] Gu S H, Zhang L, Zuo W M, Feng X C. Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014. 2862−2869
    [38] Mohanaiah P, Sathyanarayana P, GuruKumar L. Image texture feature extraction using GLCM approach. International Journal of Scientific and Research Publications, 2013, 3(5): 1−5
    [39] Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7): 971−987 doi: 10.1109/TPAMI.2002.1017623
    [40] Dalal N, Triggs B. Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). San Diego, USA: IEEE, 2005. 886−893
    [41] Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91−110 doi: 10.1023/B:VISI.0000029664.99615.94
    [42] Lindeberg T. Scale invariant feature transform. Scholarpedia, 2012, 7(5): Article No. 10491 doi: 10.4249/scholarpedia.10491
    [43] Oren M, Papageorgiou C, Sinha P, Osuna E, Poggio T. Pedestrian detection using wavelet templates. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Juan, USA: IEEE, 1997. 193−199
    [44] Lienhart R, Maydt J. An extended set of Haar-like features for rapid object detection. In: Proceedings of the International Conference on Image Processing. Rochester, USA: IEEE, 2002.
    [45] Wang X W, Ding X Q, Liu C S. Gabor filters-based feature extraction for character recognition. Pattern Recognition, 2005, 38(3): 369−379 doi: 10.1016/j.patcog.2004.08.004
    [46] Raheja J L, Kumar S, Chaudhary A. Fabric defect detection based on GLCM and Gabor filter: A comparison. Optik, 2013, 124(23): 6469−6474 doi: 10.1016/j.ijleo.2013.05.004
    [47] Siew L H, Hodgson R M, Wood E J. Texture measures for carpet wear assessment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1988, 10(1): 92−105 doi: 10.1109/34.3870
    [48] Liu K, Wang H Y, Chen H Y, Qu E Q, Tian Y, Sun H X. Steel surface defect detection using a new Haar–Weibull-Variance model in unsupervised manner. IEEE Transactions on Instrumentation and Measurement, 2017, 66(10): 2585−2596 doi: 10.1109/TIM.2017.2712838
    [49] Pernkopf F, O'Leary P. Image acquisition techniques for automatic visual inspection of metallic surfaces. NDT & E International, 2003, 36(8): 609−617
    [50] Kaggle. Severstal: Steel defect detection [Online], available: https://www.kaggle.com/c/severstal-steel-defect-detection, March 19, 2023
    [51] Qi S, Yang J, Zhong Z. A review on industrial surface defect detection based on deep learning technology. In: Proceedings of the 3rd International Conference on Machine Learning and Machine Intelligence. Hangzhou, China: ACM, 2020. 24−30
    [52] 郑凯, 李建胜. 基于深度神经网络的图像语义分割综述. 测绘与空间地理信息, 2020, 43(10): 119−125 doi: 10.3969/j.issn.1672-5867.2020.10.032

    Zheng Kai, Li Jian-Sheng. An overview of image semantic segmentation based on deep learning. Geomatics & Spatial Information Technology, 2020, 43(10): 119−125 doi: 10.3969/j.issn.1672-5867.2020.10.032
    [53] Belongie S, Malik J, Puzicha J. Shape context: A new descriptor for shape matching and object recognition. In: Proceedings of the 13th International Conference on Neural Information Processing Systems. Denver, USA: MIT Press, 2000. 798−804
    [54] Qi C R, Yi L, Su H, Guibas L J. PointNet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 652−660
    [55] Qi C R, Yi L, Su H, Guibas L J. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: Curran Associates Inc., 2017. 5105−5114
    [56] Ma X, Qin C, You H X, Ran H X, Fu Y. Rethinking network design and local geometry in point cloud: A simple residual MLP framework. In: Proceedings of the 10th International Conference on Learning Representations (ICLR). OpenReview.net, 2022.
    [57] Qian G C, Li Y C, Peng H W, Mai J J, Al Kader Hammoud H A, Elhoseiny M, et al. PointNeXt: Revisiting PointNet++ with improved training and scaling strategies. In: Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS). 2022. 23192−23204
    [58] Jain P, Tyagi V. LAPB: Locally adaptive patch-based wavelet domain edge-preserving image denoising. Information Sciences, 2015, 294: 164−181 doi: 10.1016/j.ins.2014.09.060
    [59] Li A B, Ma H W, Xu S H. Three-dimensional morphology and watershed-algorithm-based method for pitting corrosion evaluation. Buildings, 2022, 12(11): Article No. 1908 doi: 10.3390/buildings12111908
    [60] 郭皓然, 邵伟, 周阿维, 杨宇祥, 刘凯斌. 全局阈值自适应的高亮金属表面缺陷识别新方法. 仪器仪表学报, 2017, 38(11): 2797−2804

    Guo Hao-Ran, Shao Wei, Zhou A-Wei, Yang Yu-Xiang, Liu Kai-Bin. Novel defect recognition method based on adaptive global threshold for highlight metal surface. Chinese Journal of Scientific Instrument, 2017, 38(11): 2797−2804
    [61] 魏爱东. 基于脉冲涡流热成像的钢板缺陷检测研究. 电子测试, 2020, 34(7): 56−59 doi: 10.3969/j.issn.1000-8519.2020.07.020

    Wei Ai-Dong. Thermal image defect extraction based on two image segmentation algorithms. Electronic Test, 2020, 34(7): 56−59 doi: 10.3969/j.issn.1000-8519.2020.07.020
    [62] Prabha P A, Bharathwaj M, Dinesh K, Prashath G H. Defect detection of industrial products using image segmentation and saliency. Journal of Physics: Conference Series, 2021, 1916: Article No. 012165
    [63] Zhang X W, Ding Y Q, Lv Y Y, Shi A Y, Liang R Y. A vision inspection system for the surface defects of strongly reflected metal based on multi-class SVM. Expert Systems With Applications, 2011, 38(5): 5930−5939 doi: 10.1016/j.eswa.2010.11.030
    [64] Ghosh D, Kaabouch N. A survey on image mosaicing techniques. Journal of Visual Communication and Image Representation, 2016, 34: 1−11 doi: 10.1016/j.jvcir.2015.10.014
    [65] Lu R S, Shi Y Q, Li Q, Yu Q P. AOI techniques for surface defect inspection. In: Proceedings of the Applied Mechanics and Materials. Trans Tech Publications Ltd, 2010, 36. 297−302
    [66] Kong Q, Wu Z, Song Y. Online detection of external thread surface defects based on an improved template matching algorithm. Measurement, 2022, 195: 111087 doi: 10.1016/j.measurement.2022.111087
    [67] Pang G, Shen C, Cao L, Hengel V A D. Deep learning for anomaly detection: A review. ACM Computing Surveys, 2021, 54(2): 1−38
    [68] Konovalenko I, Maruschak P, Brevus V, Prentkovskis O. Recognition of scratches and abrasions on metal surfaces using a classifier based on a convolutional neural network. Metals, 2021, 11(4): Article No. 549 doi: 10.3390/met11040549
    [69] Jiang Q S, Tan D P, Li Y B, Ji S M, Cai C P, Zheng Q M. Object detection and classification of metal polishing shaft surface defects based on convolutional neural network deep learning. Applied Sciences, 2019, 10(1): Article No. 87 doi: 10.3390/app10010087
    [70] Tabernik D, Šela S, Skvarč J, Skočaj D. Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing, 2019, 31(3): 759−776
    [71] 杨洁. 基于深度学习的无监督图像异常模式检测与识别研究 [博士学位论文], 中国科学院大学, 中国, 2021

    Yang Jie. Unsupervised Visual Anomaly Detection and Recognition Based on Deep Learning [Ph. D. dissertation]. School of Computer and Technology, University of Chinese Academy of Sciences, China, 2021
    [72] Chetverikov D, Khenokh Y. Matching for shape defect detection. In: Proceedings of the 8th International Conference on Computer Analysis of Images and Patterns (CAIP). Ljubljana, Slovenia: Springer, 1999. 367−374
    [73] Wang L, Pavlidis T. Direct gray-scale extraction of features for character recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15(10): 1053−1067 doi: 10.1109/34.254062
    [74] Chui H, Rangarajan A. A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding, 2003, 89(2): 114−141 doi: 10.1016/S1077-3142(03)00009-2
    [75] Borgefors G. Hierarchical chamfer matching: A parametric edge matching algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1988, 10(6): 849−865 doi: 10.1109/34.9107
    [76] Zhang H J, Hu Q. Fast image matching based-on improved SURF algorithm. In: Proceedings of the International Conference on Electronics, Communications and Control (ICECC). Ningbo, China: IEEE, 2011. 1460−1463
    [77] Crispin A J, Rankov V. Automated inspection of PCB components using a genetic algorithm template-matching approach. The International Journal of Advanced Manufacturing Technology, 2007, 35(3): 293−300
    [78] Hashemi N S, Aghdam R B, Ghiasi A S B, Fatemi P. Template matching advances and applications in image analysis. arXiv preprint arXiv: 1610.07231, 2016.
    [79] Wang H Y, Zhang J W, Tian Y, Chen H Y, Sun H X, Liu K. A simple guidance template-based defect detection method for strip steel surfaces. IEEE Transactions on Industrial Informatics, 2019, 15(5): 2798−2809 doi: 10.1109/TII.2018.2887145
    [80] Pernkopf F. Detection of surface defects on raw steel blocks using Bayesian network classifiers. Pattern Analysis and Applications, 2004, 7(3): 333−342 doi: 10.1007/s10044-004-0232-3
    [81] Aghdam S R, Amid E, Imani M F. A fast method of steel surface defect detection using decision trees applied to LBP based features. In: Proceedings of the 7th IEEE Conference on Industrial Electronics and Applications (ICIEA). Singapore: IEEE, 2012. 1447−1452
    [82] Samy M P, Foong S, Soh G S, Yeo K S. Automatic optical & laser-based defect detection and classification in brick masonry walls. In: Proceedings of the IEEE Region 10 Conference (TENCON). Singapore: IEEE, 2016. 3521−3524
    [83] Li X G, Zhu J, Shi H R, Cong Z J. Surface defect detection of seals based on K-means clustering algorithm and particle swarm optimization. Scientific Programming, 2021, 2021: Article No. 3965247
    [84] Yue B, Wang Y P, Min Y Z, Zhang Z H, Wang W R, Yong J. Rail surface defect recognition method based on AdaBoost multi-classifier combination. In: Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). Lanzhou, China: IEEE, 2019. 391−396
    [85] Soukup D, Huber-Mörk R. Convolutional neural networks for steel surface defect detection from photometric stereo images. In: Proceedings of the 10th International Symposium on Visual Computing (ISVC). Las Vegas, USA: Springer, 2014. 668−677
    [86] Masci J, Meier U, Ciresan D, Schmidhuber J, Fricout G. Steel defect classification with max-pooling convolutional neural networks. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN). Brisbane, Australia: IEEE, 2012. 1−6
    [87] Staar B, Lütjen M, Freitag M. Anomaly detection with convolutional neural networks for industrial surface inspection. Procedia CIRP, 2019, 79: 484−489 doi: 10.1016/j.procir.2019.02.123
    [88] Liao S C, Shao L. Graph sampling based deep metric learning for generalizable person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE, 2022. 7349−7358
    [89] Alzu'bi A, Albalas F, Al-Hadhrami T, Younis L B, Bashayreh A. Masked face recognition using deep learning: A review. Electronics, 2021, 10(21): Article No. 2666 doi: 10.3390/electronics10212666
    [90] Mordia R, Verma A K. Visual techniques for defects detection in steel products: A comparative study. Engineering Failure Analysis, 2022, 134: Article No. 106047 doi: 10.1016/j.engfailanal.2022.106047
    [91] Kim M S, Park T, Park P. Classification of steel surface defect using convolutional neural network with few images. In: Proceedings of the 12th Asian Control Conference (ASCC). Kitakyushu, Japan: IEEE, 2019. 1398−1401
    [92] Wu S L, Wu Y B, Cao D H, Zheng C Y. A fast button surface defect detection method based on Siamese network with imbalanced samples. Multimedia Tools and Applications, 2019, 78(24): 34627−34648 doi: 10.1007/s11042-019-08042-w
    [93] Li X, Yang X, Ma Z, Xue J H. Deep metric learning for few-shot image classification: A selective review. arXiv preprint arXiv: 2105.08149, 2021.
    [94] Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014. 580−587
    [95] He K M, Zhang X Y, Ren S Q, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904−1916 doi: 10.1109/TPAMI.2015.2389824
    [96] Girshick R. Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015. 1440−1448
    [97] Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137−1149 doi: 10.1109/TPAMI.2016.2577031
    [98] He K M, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017. 2980−2988
    [99] Cai Z W, Vasconcelos N. Cascade R-CNN: Delving into high quality object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 6154−6162
    [100] Guo F, Qian Y, Rizos D, Suo Z, Chen X B. Automatic rail surface defects inspection based on Mask R-CNN. Transportation Research Record: Journal of the Transportation Research Board, 2021, 2675(11): 655−668 doi: 10.1177/03611981211019034
    [101] Xu Y Y, Li D W, Xie Q, Wu Q Y, Wang J. Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN. Measurement, 2021, 178: Article No. 109316 doi: 10.1016/j.measurement.2021.109316
    [102] Fang J T, Tan X Y, Wang Y H. ACRM: Attention cascade R-CNN with Mix-NMS for metallic surface defect detection. In: Proceedings of the 25th International Conference on Pattern Recognition (ICPR). Milan, Italy: IEEE, 2021. 423−430
    [103] Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016. 779−788
    [104] Bochkovskiy A, Wang C Y, Liao H Y M. YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv: 2004.10934, 2020.
    [105] Li M J, Wang H, Wan Z B. Surface defect detection of steel strips based on improved YOLOv4. Computers and Electrical Engineering, 2022, 102: Article No. 108208 doi: 10.1016/j.compeleceng.2022.108208
    [106] Usamentiaga R, Lema D G, Pedrayes O D, Garcia D F. Automated surface defect detection in metals: A comparative review of object detection and semantic segmentation using deep learning. IEEE Transactions on Industry Applications, 2022, 58(3): 4203−4213 doi: 10.1109/TIA.2022.3151560
    [107] Xu Y H, Wang X J, Li S Y. Track surface defect detection based on EfficientDet. In: Proceedings of the 5th International Conference on Electrical Engineering and Information Technologies for Rail Transportation (EITRT) 2021: Rail Transportation Information Processing and Operational Management Technologies. Singapore: Springer, 2022. 56−66
    [108] Wang C Y, Bochkovskiy A, Liao H Y M. Scaled-YOLOv4: Scaling cross stage partial network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2021. 13024−13033
    [109] He Y, Song K C, Meng Q G, Yan Y H. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Transactions on Instrumentation and Measurement, 2020, 69(4): 1493−1504 doi: 10.1109/TIM.2019.2915404
    [110] Anthony A, Ho E S L, Woo W L, Gao B. A review and benchmark on state-of-the-art steel defects detection [Online], available: http://dx.doi.org/10.2139/ssrn.4121951, March 19, 2023
    [111] Song G L, Liu Y, Wang X G. Revisiting the sibling head in object detector. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 11560−11569
    [112] Lin T Y, Goyal P, Girshick R, He K M, Dollár P. Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017. 2999−3007
    [113] Ma P F, Ma J, Wang X J, Yang L C, Wang N N. Deformable convolutional networks for multi-view 3D shape classification. Electronics Letters, 2018, 54(24): 1373−1375 doi: 10.1049/el.2018.6851
    [114] Law H, Deng J. CornerNet: Detecting objects as paired keypoints. In: Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018. 765−781
    [115] Ge Z, Liu S T, Wang F, Li Z M, Sun J. YOLOX: Exceeding YOLO series in 2021. arXiv preprint arXiv: 2107.08430, 2021.
    [116] Zhu B J, Wang J F, Jiang Z K, Zong F H, Liu S T, Li Z M, et al. AutoAssign: Differentiable label assignment for dense object detection. arXiv preprint arXiv: 2007.03496, 2020.
    [117] Lu H T, Fang M Y, Qiu Y, Xu W Q. An anchor-free defect detector for complex background based on pixelwise adaptive multiscale feature fusion. IEEE Transactions on Instrumentation and Measurement, 2023, 72: Article No. 5002312
    [118] Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S. End-to-end object detection with transformers. In: Proceedings of the 16th European Conference on Computer Vision (ECCV). Glasgow, UK: Springer, 2020. 213−229
    [119] Zhu X, Su W, Lu L, Li B, Wang X, Dai J. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv: 2010.04159, 2020.
    [120] Misra I, Girdhar R, Joulin A. An end-to-end transformer model for 3D object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, Canada: IEEE, 2021. 2886−2897
    [121] Lv X, Duan F, Jiang J, Fu X, Gan L. Deep metallic surface defect detection: The new benchmark and detection network. Sensors, 2020, 20(6): Article No. 1562
    [122] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE, 2015. 3431−3440
    [123] Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Munich, Germany: Springer, 2015. 234−241
    [124] Sabet D N, Zarifi M R, Khoramdel J, Borhani Y, Najafi E. An automated visual defect segmentation for flat steel surface using deep neural networks. In: Proceedings of the 12th International Conference on Computer and Knowledge Engineering (ICCKE). Mashhad, Islamic Republic of Iran: IEEE, 2022. 423−427
    [125] Lin T Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 2117−2125
    [126] Xie S N, Girshick R, Dollár P, Tu Z W, He K M. Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 5987−5995
    [127] Koonce B. EfficientNet. Convolutional Neural Networks With Swift for Tensorflow: Image Recognition and Dataset Categorization. Berkeley: Apress, 2021. 109−123
    [128] Chen L C, Papandreou G, Kokkinos I, Murphy K, Yuille A L. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834−848 doi: 10.1109/TPAMI.2017.2699184
    [129] Zheng S, Jayasumana S, Romera-Paredes B, Vineet V, Su Z Z, Du D L, et al. Conditional random fields as recurrent neural networks. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015. 1529−1537
    [130] Ma L F, Li J. SD-GCN: Saliency-based dilated graph convolution network for pavement crack extraction from 3D point clouds. International Journal of Applied Earth Observation and Geoinformation, 2022, 111: Article No. 102836 doi: 10.1016/j.jag.2022.102836
    [131] Huang Y B, Qiu C Y, Guo Y, Wang X N, Yuan K. Surface defect saliency of magnetic tile. In: Proceedings of the 14th International Conference on Automation Science and Engineering (CASE). Munich, Germany: IEEE, 2018. 612−617
    [132] Tian H, Li F. Autoencoder-based fabric defect detection with cross-patch similarity. In: Proceedings of the 16th International Conference on Machine Vision Applications (MVA). Tokyo, Japan: IEEE, 2019. 1−6
    [133] Mei S, Yang H, Yin Z P. An unsupervised-learning-based approach for automated defect inspection on textured surfaces. IEEE Transactions on Instrumentation and Measurement, 2018, 67(6): 1266−1277 doi: 10.1109/TIM.2018.2795178
    [134] Huang C Q, Cao J K, Ye F, Li M S, Zhang Y, Lu C W. Inverse-transform Autoencoder for anomaly detection. arXiv preprint arXiv: 1911.10676, 2019.
    [135] Zimmerer D, Petersen J, Kohl S A A, Maier-Hein K H. A case for the score: Identifying image anomalies using variational autoencoder gradients. arXiv preprint arXiv: 1912.00003, 2019.
    [136] Kwon G, Prabhushankar M, Temel D, AlRegib G. Backpropagated gradient representations for anomaly detection. In: Proceedings of the 16th European Conference on Computer Vision (ECCV). Glasgow, UK: Springer, 2020. 206−226
    [137] Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial networks. Communications of the ACM, 2020, 63(11): 139−144 doi: 10.1145/3422622
    [138] Baur C, Denner S, Wiestler B, Navab N, Albarqouni S. Autoencoders for unsupervised anomaly segmentation in brain MR images: A comparative study. Medical Image Analysis, 2021, 69: Article No. 101952 doi: 10.1016/j.media.2020.101952
    [139] Schlegl T, Seeböck P, Waldstein S M, Langs G, Schmidt-Erfurth U. f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks. Medical Image Analysis, 2019, 54: 30−44 doi: 10.1016/j.media.2019.01.010
    [140] Rudolph M, Wehrbein T, Rosenhahn B, Wandt B. Fully convolutional cross-scale-flows for image-based defect detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa, USA: IEEE, 2022. 1829−1838
    [141] Yu J W, Zheng Y, Wang X, Li W, Wu Y S, Zhao R, et al. FastFlow: Unsupervised anomaly detection and localization via 2D normalizing flows. arXiv preprint arXiv: 2111.07677, 2021.
    [142] Yi J H, Yoon S. Patch SVDD: Patch-level SVDD for anomaly detection and segmentation. In: Proceedings of the 15th Asian Conference on Computer Vision (ACCV). Kyoto, Japan: Springer, 2021. 375−390
    [143] Tax D M J, Duin R P W. Support vector data description. Machine Learning, 2004, 54(1): 45−66 doi: 10.1023/B:MACH.0000008084.60811.49
    [144] Yang M H, Wu P, Feng H. MemSeg: A semi-supervised method for image surface defect detection using differences and commonalities. Engineering Applications of Artificial Intelligence, 2023, 119: Article No. 105835 doi: 10.1016/j.engappai.2023.105835
    [145] Roth K, Pemula L, Zepeda J, Schölkopf B, Brox T, Gehler P. Towards total recall in industrial anomaly detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE, 2022. 14298−14308
    [146] Bae J, Lee J H, Kim S. Image anomaly detection and localization with position and neighborhood information. arXiv preprint arXiv: 2211.12634, 2022.
    [147] Bergmann P, Fauser M, Sattlegger D, Steger C. Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 4182−4191
    [148] Batzner K, Heckler L, König R. EfficientAD: Accurate visual anomaly detection at millisecond-level latencies. arXiv preprint arXiv: 2303.14535, 2023.
  • 加载中
图(16) / 表(3)
计量
  • 文章访问数:  2057
  • HTML全文浏览量:  2463
  • PDF下载量:  302
  • 被引次数: 0
出版历程
  • 收稿日期:  2023-02-06
  • 录用日期:  2023-05-18
  • 网络出版日期:  2023-07-03

目录

    /

    返回文章
    返回