2.624

2020影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于深度学习的桥梁裂缝检测算法研究

李良福 马卫飞 李丽 陆铖

李良福, 马卫飞, 李丽, 陆铖. 基于深度学习的桥梁裂缝检测算法研究. 自动化学报, 2019, 45(9): 1727-1742. doi: 10.16383/j.aas.2018.c170052
引用本文: 李良福, 马卫飞, 李丽, 陆铖. 基于深度学习的桥梁裂缝检测算法研究. 自动化学报, 2019, 45(9): 1727-1742. doi: 10.16383/j.aas.2018.c170052
LI Liang-Fu, MA Wei-Fei, LI Li, LU Cheng. Research on Detection Algorithm for Bridge Cracks Based on Deep Learning. ACTA AUTOMATICA SINICA, 2019, 45(9): 1727-1742. doi: 10.16383/j.aas.2018.c170052
Citation: LI Liang-Fu, MA Wei-Fei, LI Li, LU Cheng. Research on Detection Algorithm for Bridge Cracks Based on Deep Learning. ACTA AUTOMATICA SINICA, 2019, 45(9): 1727-1742. doi: 10.16383/j.aas.2018.c170052

基于深度学习的桥梁裂缝检测算法研究

doi: 10.16383/j.aas.2018.c170052
基金项目: 

中央高校基本科研业务费专项资金 GK201703056

国家自然科学基金 61401263

国家自然科学基金 61573232

详细信息
    作者简介:

    李良福 陕西师范大学计算机学院副教授.2006年获得西安交通大学博士学位, 美国卡内基梅隆大学机器人研究所访问学者.主要研究方向为计算机视觉, 深度学习, 人工智能.E-mail:longford@xjtu.edu.cn

    李丽 陕西师范大学计算机科学学院硕士研究生.主要研究方向为数字图像处理, 计算机视觉, 深度学习.E-mail:lili94815@outlook.com

    陆铖 陕西师范大学计算机学院副教授.2013年获得加拿大阿尔伯塔大学博士学位.主要研究方向为图像处理, 模式识别.E-mail:chenglu@snnu.edu.cn

    通讯作者:

    马卫飞 陕西师范大学计算机科学学院硕士研究生.主要研究方向为数字图像处理, 计算机视觉, 深度学习, 人工智能.本文通信作者.E-mail:weifei@snnu.edu.cn

Research on Detection Algorithm for Bridge Cracks Based on Deep Learning

Funds: 

The Fundamental Research Funds for the Central Universities GK201703056

National Natural Science Foundation of China 61401263

National Natural Science Foundation of China 61573232

More Information
    Author Bio:

    Associate professor at the College of Computer Science, Shaanxi Normal University. He received his Ph. D. degree from Xi'an Jiao Tong University in 2006. He was a visiting scholar at the Robotics Institute, Carnegie Mellon University, USA. His research interest covers computer vision, deep learning, artificial intelligence

    Master student at the College of Computer Science, Shaanxi Normal University. Her research interest covers digital image processing, computer vision, deep learning

    Associate professor at the College of Computer Science, Shaanxi Normal University. He received his Ph. D degree from University of Alberta, Canada in 2013. His research interest covers image processing, pattern recognition

    Corresponding author: MA Wei-Fei Master student at the College of Computer Science, Shaanxi Normal University. His research interest covers digital image processing, computer vision, deep learning, artificial intelligence. Corresponding author of this paper
  • 摘要: 传统的图像处理算法不能很好地对桥梁裂缝进行检测,而经典的深度学习模型直接用于桥梁裂缝的检测,效果不甚理想.针对这些问题,本文提出了一种基于深度学习的桥梁裂缝检测算法.首先,利用滑动窗口算法将桥梁裂缝图像切分为较小的桥梁裂缝面元图像和桥梁背景面元图像,并根据对面元图像的分析,提出一种基于卷积神经网络(Convolutional neural networks,CNN)的DBCC(Deep bridge crack classify)分类模型,用于桥梁背景面元和桥梁裂缝面元的识别.然后,基于DBCC分类模型结合改进的窗口滑动算法对桥梁裂缝进行检测.最后,采用图像金字塔和感兴趣区域(Region of interest,ROI)结合的搜索策略对算法进行加速.实验结果表明:与传统算法相比,本文算法具有更好的识别效果和更强的泛化能力.
    1)  本文责任编委 王亮
  • 图  1  经典深度学习模型和桥梁裂缝特点示意图

    Fig.  1  Schematic diagram of classical depth learning model and bridge crack characteristics

    图  2  桥梁裂缝面元数据集人工扩增方式示意图

    Fig.  2  Schematic diagram of manual expansion of bridge crack surface metadata set

    图  3  桥梁裂缝图像特点示意图

    Fig.  3  Image characteristics of bridge cracks

    图  4  桥梁裂缝面元和背景面元示意图

    Fig.  4  Schematic diagram of bridge crack surface and background surface

    图  5  CIFAR10模型和DBCC模型检测结果的示意图

    Fig.  5  Detection results of CIFAR10 and DBCC model

    图  6  DBCC模型的网络结构示意图

    Fig.  6  Network structure of DBCC model

    图  7  卷积层可视化结果示意图

    Fig.  7  Visualization results of convolution layer

    图  8  改进的窗口滑动算法示意图

    Fig.  8  Improved window sliding algorithm

    图  9  图像金字塔和ROI加速策略示意图

    Fig.  9  Schematic diagram of image pyramid and ROI acceleration strategy

    图  10  DBCC模型桥梁裂缝检测结果的示意图

    Fig.  10  Schematic diagram of DBCC model bridge crack detection results

    图  11  基于桥梁裂缝面元的裂缝提取和定位算法流程示意图

    Fig.  11  Flow chart of crack extraction and location algorithm based on bridge crack surface element

    图  12  CIFAR10模型和DBCC模型对于桥梁裂缝检测结果的对比

    Fig.  12  Comparison between CIFAR10 model and DBCC model for bridge crack detection

    图  13  概率区分阈值Tp对于桥梁裂缝识别效果的影响

    Fig.  13  Effect of probability discrimination threshold Tp on bridge crack identification

    图  14  基于桥梁裂缝面元的桥梁裂缝提取算法对于检测结果的影响

    Fig.  14  Influence of the bridge crack extraction algorithm based on bridge crack surface element on detection

    图  15  各种裂缝检测算法对于桥梁裂缝定位准确度的影响

    Fig.  15  Influence of various crack detection algorithms on accuracy of bridge crack location

    图  16  主流裂缝检测算法和本文算法对于桥梁裂缝检测的效果图

    Fig.  16  Detection results of the main stream crack detection algorithm and our algorithm for bridge cracks image

    图  17  基于本文算法进行桥梁裂缝检测的部分结果

    Fig.  17  Partial results of bridge crack detection based on our algorithm

    表  1  DBCC模型的输入层至第2池化层各层的具体模型构建参数

    Table  1  Modeling parameters from the input layer to the second pool layer of the DBCC model

    模型 输入层 卷积层1 Max-Pooling 1 卷积层2 Ave-Pooling 2
    CIFAR10 32 × 32 × 3 Conv5-1-2-32 MP3-2-0-32 Conv5-1-2-32 AVE3-2-0-32
    DBCC-A 16 × 16 × 3 Conv3-1-1-32 MP3-2-0-32 Conv3-1-1-32 AVE3-2-0-32
    DBCC-B 16 × 16 × 3 Conv5-1-2-32 MP3-2-0-32 Conv5-1-2-32 AVE3-2-0-32
    DBCC-C 16 × 16 × 3 Conv5-1-2-32 MP2-2-0-32 Conv5-1-2-32 AVE2-2-0-32
    DBCC-D 16 × 16 × 3 Conv5-1-2-32 MP2-2-0-32 Conv5-1-2-64 AVE2-2-0-64
    DBCC-E 16 × 16 × 3 Conv5-1-2-32-LRN MP2-2-0-32 Conv5-1-2-64 AVE2-2-0-64
    DBCC 16 × 16 × 3 Conv5-1-2-32-LRN MP2-2-0-32 Conv5-1-2-64 AVE2-2-0-64
    下载: 导出CSV

    表  2  DBCC模型的第3卷积层至输出层各层的具体模型构建参数

    Table  2  Modeling parameters from the third volume accumulated layer to the output layer of the DBCC model

    模型 卷积层3 Ave-Pooling 3 卷积层4 FC1 Dropout层 FC2 输出层
    CIFAR10 Conv5-1-2-64 AVE3-2-0-64 --- 2 --- --- Softmax
    DBCC-A Conv3-1-1-64 AVE3-2-0-64 --- 2 --- --- Softmax
    DBCC-B Conv5-1-2-64 AVE3-2-0-64 --- 2 --- --- Softmax
    DBCC-C Conv5-1-2-64 AVE2-2-0-64 Conv2-1-0-64 2 --- --- Softmax
    DBCC-D Conv5-1-2-128 AVE2-2-0-128 Conv2-1-0-256 128 --- 2 Softmax
    DBCC-E Conv5-1-2-128 AVE2-2-0-128 Conv2-1-0-256 128 --- 2 Softmax
    DBCC Conv5-1-2-128 AVE2-2-0-128 Conv2-1-0-256 128 Dropout 2 Softmax
    下载: 导出CSV

    表  3  人工数据集扩增方法对于DBCC模型识别准确率的影响

    Table  3  Effect of artificial data set amplification on recognition accuracy of DBCC model

    桥梁裂缝面元的总数 有无数据集的扩增 DBCC模型正确识别数 准确率
    500 353 70.6
    500 489 97.8
    下载: 导出CSV

    表  4  各模型对于桥梁裂缝面元识别的准确率

    Table  4  Accuracy of each model for bridge crack surface element identification

    行号 模型 裂缝面元的总数 裂缝面元规格(像素) 正确识别的裂缝面元数(幅) 识别的准确率(%)
    0 CIFAR10 1 000 8×8 34 3.4
    1 CIFAR10 1 000 16 × 16 347 34.7
    2 CIFAR10 1 000 32 × 32 976 97.6
    3 DBCC-A 1 000 16 × 16 954 95.4
    4 DBCC-B 1 000 16 × 16 958 95.8
    5 DBCC-C 1 000 16 × 16 960 96.0
    6 DBCC-D 1 000 16 × 16 970 97.0
    7 DBCC-E 1 000 16 × 16 973 97.3
    8 DBCC 1 000 8 × 8 48 4.8
    9 DBCC 1 000 16 × 16 979 97.9
    下载: 导出CSV

    表  5  算法加速策略对于本文识别算法的影响

    Table  5  Effect of algorithm acceleration strategy on the recognition algorithm in this paper

    桥梁裂缝图像的编号 未采用加速策略耗时(s) 采用加速策略耗时(s)
    1 16.976065 3.345511
    2 17.332235 3.324066
    3 16.834522 3.295553
    4 17.793048 3.500678
    5 17.123332 3.349923
    下载: 导出CSV

    表  6  桥梁裂缝定位的位置坐标

    Table  6  Position coordinates of bridge crack location

    图像的编号 左上角点坐标(像素) 右下角点坐标(像素)
    1 (239, 0) (381, 512)
    2 (0, 98) (512, 370)
    3 (0, 33) (512, 385)
    4 (218, 0) (294, 512)
    下载: 导出CSV

    表  7  桥梁裂缝定位准确度的量化分析

    Table  7  Quantitative analysis of location accuracy of bridge cracks

    算法 S < 10像素的准确率(%) S < 20像素的准确率(%) S < 30像素的准确率(%) 平均运行时间(s)
    迭代阈值分割算法[3] 10.4 20.0 27.4 0.998
    FFA算法[11] 13.4 27.1 34.8 3.976
    最小路径选择算法[8] 17.6 37.6 61.0 11.493
    本文算法 46.6 67.4 84.4 3.468
    下载: 导出CSV

    表  8  桥梁裂缝检测提取算法的量化分析对比

    Table  8  Quantitative analysis and comparison of bridge crack detection and extraction algorithm

    算法 迭代阈值分割算法 FFA算法 最小路径选择算法 本文算法
    评价指标 Pre/Rec Pre/Rec Pre/Rec Pre/Rec
    第1幅图像 15.7 %/16.3 % 89.3 %/91.5 % 90.3 %/91.5 % 97.5 %/98.7
    第2幅图像 12.5 %/9.5 % 67.5 %/83.5 % 73.5 %/80.4 % 92.39 %/94.6
    第3幅图像 5.6 %/5.57 % 90.5 %/90.3 % 63.3 %/64.39 % 94.5 %/96.3
    第4幅图像 7.8 %/7.5 % 18.3 %/21.7 % 35.7 %/20.3 % 91.31 %/92.5
    第5幅图像 10.1 %/13.4 % 73.2 %/63.7 % 91.2 %/93.5 % 91.6 %/90.4
    平均运行时间(s) 0.936 3.896 11.314 3.357
    下载: 导出CSV
  • [1] 马建, 孙守增, 杨琦.中国桥梁工程学术研究综述:2014.中国公路学报, 2014, 27(5):1-96 http://d.old.wanfangdata.com.cn/Periodical/zgglxb201405001

    Ma Jian, Sun Shou-Zeng, Yang Qi. Review on China's bridge engineering research:2014. China Journal of Highway and Transport, 2014, 27(5):1-96 http://d.old.wanfangdata.com.cn/Periodical/zgglxb201405001
    [2] 魏武, 王俊杰, 蔡钊雄.基于小波和Radon变换的桥梁裂缝检测.计算机工程与设计, 2013, 34(9):3151-3157 http://d.old.wanfangdata.com.cn/Periodical/jsjgcysj201309031

    Wei Wu, Wang Jun-Jie, Cai Zhao-Xiong. Bridge crack detection based on wavelet and Radon transform. Computer Engineering and Design, 2013, 34(9):3151-3157 http://d.old.wanfangdata.com.cn/Periodical/jsjgcysj201309031
    [3] Oh H, Garrick N W, Achenie L E K. Segmentation algorithm using iterative clipping for processing noisy pavement images. In:Proceedings of the 2nd International Conference on Imaging Technologies:Techniques and Applications in Civil Engineering. Davos, Switzerland:TRID, 1998. 138-147 https://cedb.asce.org/CEDBsearch/record.jsp?dockey=0109703
    [4] Li Q Q, Liu X L. Novel approach to pavement image segmentation based on neighboring difference histogram method. In:Proceedings of the 2008 Congress on Image and Signal Processing. Sanya, Hainan, China:IEEE, 2008. 792-796 https://ieeexplore.ieee.org/document/4566413
    [5] Landstrom A, Thurley M J. Morphology-based crack detection for steel slabs. IEEE Journal of Selected Topics in Signal Processing, 2012, 6(7):866-875 doi: 10.1109/JSTSP.2012.2212416
    [6] Varadharajan S, Jose S, Sharma K, Wander L, Mertz C. Vision for road inspection. In:Proceedings of the 2014 IEEE Winter Conference on Applications of Computer Vision (WACV). Steamboat Springs, CO, USA:IEEE, 2014. 115-122 https://ieeexplore.ieee.org/document/6836111
    [7] Jahanshahi M R, Masri S F, Padgett C W, Sukhatme G S. An innovative methodology for detection and quantification of cracks through incorporation of depth perception. Machine Vision and Applications, 2013, 24(2):227-241 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=c9ba111ad8b36c3b26d43287663c1fdb
    [8] Amhaz R, Chambon S, Idier J, Baltazart V. Automatic crack detection on two-dimensional pavement images:an algorithm based on minimal path selection. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(10):2718-2729 doi: 10.1109/TITS.2015.2477675
    [9] Zou Q, Cao Y, Li Q Q, Mao Q Z, Wang S. CrackTree:automatic crack detection from pavement images. Pattern Recognition Letters, 2012, 33(3):227-238 http://d.old.wanfangdata.com.cn/NSTLQK/NSTL_QKJJ0226254615/
    [10] Oliveira H, Correia P L. Automatic road crack detection and characterization. IEEE Transactions on Intelligent Transportation Systems, 2013, 14(1):155-168 http://d.old.wanfangdata.com.cn/Periodical/jsjfzsjytxxxb201409008
    [11] Nguyen T S, Begot S, Duculty F, Avila M. Free-form anisotropy:a new method for crack detection on pavement surface images. In:Proceedings of the 18th IEEE International Conference on Image Processing (ICIP). Brussels, Belgium:IEEE, 2011. 1069-1072 https://ieeexplore.ieee.org/document/6115610
    [12] Mustafa R, Mohamed E A. Concrete crack detection based multi-block CLBP features and SVM classifier. Journal of Theoretical and Applied Information Technology, 2015, 81(1):151-160
    [13] Nguyen H N, Kam T Y, Cheng P Y. An automatic approach for accurate edge detection of concrete crack utilizing 2D geometric features of crack. Journal of Signal Processing Systems, 2014, 77(3):221-240 doi: 10.1007/s11265-013-0813-8
    [14] Arena A, Piane C D, Sarout J. A new computational approach to cracks quantification from 2D image analysis:application to micro-cracks description in rocks. Computers & Geosciences, 2014, 66:106-120 https://www.sciencedirect.com/science/article/pii/S0098300414000107
    [15] Quintana M, Torres J, Menéndez J M. A simplified computer vision system for road surface inspection and maintenance. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(3):608-619 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=e6dd7d675797cd3abc3bdd685ed72d27
    [16] Shi Y, Cui L M, Qi Z Q, Meng F, Chen Z S. Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12):3434-3445 doi: 10.1109/TITS.2016.2552248
    [17] Zhang D J, Li Q Q, Chen Y, Cao M, He L, Zhang B L. An efficient and reliable coarse-to-fine approach for asphalt pavement crack detection. Image and Vision Computing, 2017, 57:130-146 doi: 10.1016/j.imavis.2016.11.018
    [18] Hassan N, Mathavan S, Kamal K. Road crack detection using the particle filter. In:Proceedings of the 23rd International Conference on Automation and Computing. Huddersfield, UK:IEEE, 2017. 1-6
    [19] Oliveira H, Correia P L. Road surface crack detection:Improved segmentation with pixel-based refinement. In:Proceedings of the 25th European Signal Processing Conference. Kos, Greece:IEEE, 2017. 2026-2030 https://ieeexplore.ieee.org/document/8081565/
    [20] Simonyan K, Zisserman A. Two-stream convolutional networks for action recognition in videos. In:Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada:MIT Press, 2014. 568-576 http://www.oalib.com/paper/4082156
    [21] Zeiler M D, Fergus R. Visualizing and understanding convolutional networks. In:Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland:Springer, 2014. 818-833 doi: 10.1007%2F978-3-319-10590-1_53
    [22] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In:Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada:Curran Associates Inc., 2012. 1097-1105 https://dl.acm.org/citation.cfm?id=2999257
    [23] Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In:Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA:IEEE, 2015. 1-9 http://www.oalib.com/paper/4068635#.XY1zPvngmMs
    [24] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In:Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA:IEEE, 2016. 2818-2826 https://arxiv.org/abs/1512.00567v2
    [25] Girshick R. Fast R-CNN. In:Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile:IEEE, 2015. 1440-1448 https://ieeexplore.ieee.org/document/7410526?arnumber=7410526
    [26] Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN:towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149 doi: 10.1109/TPAMI.2016.2577031
    [27] Krizhevsky A. Convolutional deep belief networks on cifar-10[Online], available: http://www.cs.toronto.edu//kriz//conv-cifar10-aug2010.pdf, Aygyst 1, 2010
    [28] Lin M, Chen Q, Yan S C. Network in network. arXiv PreprintarXiv: 1312.4400, 2014 http://www.oalib.com/paper/4042246#.XY1zuPngmMs
    [29] Hinton G E, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov R R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv Preprint, arXiv: 1207.0580, 2012
  • 加载中
图(17) / 表(8)
计量
  • 文章访问数:  1858
  • HTML全文浏览量:  699
  • PDF下载量:  359
  • 被引次数: 0
出版历程
  • 收稿日期:  2017-01-20
  • 录用日期:  2018-02-07
  • 刊出日期:  2019-09-20

目录

    /

    返回文章
    返回