2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

强边缘提取网络用于非均匀运动模糊图像盲复原

黄彦宁 李伟红 崔金凯 龚卫国

黄彦宁, 李伟红, 崔金凯, 龚卫国. 强边缘提取网络用于非均匀运动模糊图像盲复原. 自动化学报, 2021, 47(11): 2637−2653 doi: 10.16383/j.aas.c190654
引用本文: 黄彦宁, 李伟红, 崔金凯, 龚卫国. 强边缘提取网络用于非均匀运动模糊图像盲复原. 自动化学报, 2021, 47(11): 2637−2653 doi: 10.16383/j.aas.c190654
Huang Yan-Ning, Li Wei-Hong, Cui Jin-Kai, Gong Wei-Guo. Strong edge extraction network for non-uniform blind motion image deblurring. Acta Automatica Sinica, 2021, 47(11): 2637−2653 doi: 10.16383/j.aas.c190654
Citation: Huang Yan-Ning, Li Wei-Hong, Cui Jin-Kai, Gong Wei-Guo. Strong edge extraction network for non-uniform blind motion image deblurring. Acta Automatica Sinica, 2021, 47(11): 2637−2653 doi: 10.16383/j.aas.c190654

强边缘提取网络用于非均匀运动模糊图像盲复原

doi: 10.16383/j.aas.c190654
基金项目: 国家科技惠民计划项目(2013GS500303), 广西科学研究与技术开发计划项目(桂科AA17129002)资助
详细信息
    作者简介:

    黄彦宁:重庆大学光电工程学院硕士研究生. 主要研究方向为图像处理技术. E-mail: 20122858@cqu.edu.cn

    李伟红:重庆大学光电工程学院教授. 主要研究方向为图像处理技术, 模式识别. 本文通信作者.E-mail: weihongli@cqu.edu.cn

    崔金凯:重庆大学光电工程学院博士研究生. 主要研究方向为图像处理技术. E-mail: jinkaicui@cqu.edu.cn

    龚卫国:重庆大学光电工程学院教授. 主要研究方向为图像处理技术.E-mail: wggong@cqu.edu.cn

Strong Edge Extraction Network for Non-uniform Blind Motion Image Deblurring

Funds: Supported by the National Science and Technolog Program for Public Wellbeing, China (2013GS500303), the Key Projects of Science and Technology Agency of Guangxi Province, China (Guike AA17129002)
More Information
    Author Bio:

    HUANG Yan-Ning Master student at the College of Optoelectronic Engineering, Chongqing University. His main research interest is image processing technology

    LI Wei-Hong Professor at the College of Optoelectronic Engineering, Chongqing University. Her research interest covers image processing technology and pattern recognition. Corresponding author of this paper

    CUI Jin-Kai Ph. D. candidate at the College of Optoelectronic Engineering, Chongqing University. His main research interest is image processing technology

    GONG Wei-Guo Professor at the College of Optoelectronic Engineering, Chongqing University. His main research interest is image processing technology

  • 摘要: 基于深度学习的非均匀运动图像去模糊方法已经获得了较好的效果. 然而, 现有的方法通常存在对边缘恢复不清晰的问题. 因此, 本文提出一种强边缘提取网络(Strong-edge extraction network, SEEN), 用于提取非均匀运动模糊图像的强边缘以提高图像边缘复原质量. 设计的强边缘提取网络由两个子网络SEEN-1和SEEN-2组成, SEEN-1实现双边滤波器的功能, 用于提取滤除了细节信息后的图像边缘. SEEN-2实现L0平滑滤波器的功能, 用于提取模糊图像的强边缘. 本文还将对应网络层提取的强边缘特征图与模糊特征图叠加, 进一步利用强边缘特征. 最后, 本文在GoPro数据集上进行了验证实验, 结果表明: 本文提出的网络可以较好地提取非均匀运动模糊图像的强边缘, 复原图像在客观和主观上都可以达到较好的效果.
  • 图  1  均匀运动模糊图像和非均匀运动模糊图像及其模糊核示意图

    Fig.  1  Uniform and non-uniform motion blurry image and their blur kernel diagram

    图  2  提出的非均匀运动模糊图像复原网络结构图

    Fig.  2  Proposed network structure for the non-uniform motion blurry image deblurring

    图  3  模糊图像强边缘提取原理示意图

    Fig.  3  Diagram of strong edge extraction of blurred image

    图  4  强边缘提取网络结构示意图

    Fig.  4  Structure diagram of strong edge extraction network

    图  5  本文提出的双方向融合梯度计算方法示意图

    Fig.  5  Diagram of gradient calculation method of two-directions fusion proposed in this paper

    图  6  改进残差模块

    Fig.  6  Improvement of residual block

    图  7  不同大小滤波器的感受野

    Fig.  7  Receptive field of different size filters

    图  8  数据集示例图像

    Fig.  8  Sample images of dataset

    图  9  有无强边缘提取网络的非均匀模糊图像复原结果比较实例 ((a) 模糊图像; (b) 模糊图像梯度图;(c) SEEN-1 输出; (d) SEEN-2 输出 (强边缘); (e) 无强边缘提取网络的复原结果;(f) 有强边缘提取网络的复原结果; (g) 清晰图像)

    Fig.  9  Comparison of restoration results of non-uniform motion blurry image with or without edge extraction network ((a) Blurry image; (b) Gradient of blurry image; (c) Output of SEEN-1; (d) Edge of SEEN-2 (Strong edge); (e) Restoration results of network without strong edge extraction network; (f) Restoration results of network with strong edge extraction network; (g) Clear image)

    图  10  强边缘提取网络中间结果分析

    Fig.  10  The intermediate results analysis of strong edge extraction network

    图  11  强边缘提取网络效果分析

    ((i)模糊图像小块(30×30); (ii)模糊图像小块边缘(30×30); (iii) SEEN-1输出;(iv) SEEN-2输出; (v)灰度分布图; (vi)列灰度值加和曲线图)

    Fig.  11  Effect analysis of strong edge extraction network

    ((i) Blurry image patch (30×30); (ii) Edge of blurry image patch (30×30); (iii) Output of SEEN-1; (iv) Output of SEEN-2; (v) Gray value distribution; (vi) Column gray value addition)

    12  对比实验结果

    12  Results of comparative experiments

    图  13  对比实验图像的PSNR和SSIM指标值柱状图

    Fig.  13  Histogram of PSNR and SSIM values of comparative experimental images

    表  1  交叉特征提取残差模块有效性验证实验结果(GoPro数据集)

    Table  1  Validation experiment results of cross-resnet block (GoPro dataset)

    评价指标残差模块交叉特征提取残差模块
    PSNR29.980030.2227
    SSIM0.88920.8944
    下载: 导出CSV

    表  2  对比实验图像的PSNR值比较

    Table  2  PSNR value comparison of comparative experimental images

    图像文献 [4]文献 [7]文献 [10]文献 [13]文献 [21]本文方法
    (i)26.363027.824829.819427.247327.258830.8864
    (ii)25.909825.336826.054022.524424.901127.0760
    (iii)22.527025.727929.416425.887726.782328.6511
    (iv)24.586827.418729.654825.738827.483730.0529
    下载: 导出CSV

    表  3  对比实验图像的SSIM值比较

    Table  3  SSIM value comparison of comparative experimental images

    图像文献 [4]文献 [7]文献 [10]文献 [13]文献 [21]本文方法
    (i)0.80310.84720.88210.78580.82900.8967
    (ii)0.79560.75930.82330.60340.74240.8335
    (iii)0.65020.76580.87380.71320.79280.8497
    (iv)0.74700.81700.87410.71250.82180.8764
    下载: 导出CSV

    表  4  复原图像的平均PSNR值和平均SSIM值(GoPro数据集)

    Table  4  Average value of PSNR and SSIM of restored images (GoPro dataset)

    评价指标文献 [4]文献 [7]文献 [10]文献 [13]文献 [21]文献 [22]文献 [23]本文方法
    PSNR(dB)24.503426.398030.100027.464824.529425.920029.550030.2227
    SSIM0.76210.82770.91370.80930.77610.78100.93400.8944
    下载: 导出CSV
  • [1] Xu L, Jia J Y. Two-phase kernel estimation for robust motion deblurring. In: Proceedings of the 2010 European Conference on Computer Vision. Crete, Greece: Springer, 2010. 157−170
    [2] Mesarovic V Z, Galatsanos N P. MAP and regularized constrained total least-squares image restoration. In: Proceedings of the 1st International Conference on Image Processing. Austin, TX, USA: IEEE, 1994. 177−181
    [3] Michael K N, Robert J P, Felipe P. A New Approach to Constrained Total Least Squares Image Restoration. Linear Algebra and its Applications, 2000, 316(1-3): 237-258 doi: 10.1016/S0024-3795(00)00115-4
    [4] Li W H, Chen Y Q, Chen R, Gong W G, Zhao B X. Hybrid order l0-regularized blur kernel estimation model for image blind deblurring. In: Proceedings of the 2017 International Symposium on Neural Networks. Sapporo, Japan: Springer, 2017. 239−247
    [5] Li W H, Chen R, Xu S W, Gong W G. Blind motion image deblurring using nonconvex higher-order total variation model. Journal of Electronic Imaging, 2016, 25(5): 053033.1-053033.19
    [6] Shen Z Y, Xu T F, Pan J S. Non-uniform Motion Deblurring with Kernel Grid Regularization. Signal Processing Image Communication, 2018, 62(0): 1-15
    [7] Chakrabarti A. A neural approach to blind motion deblurring. In: Proceedings of the 2016 European Conference on Computer Vision. Amsterdam, Netherlands: Springer, 2016. 221−235
    [8] Xu X Y, Pan J S, Zhang Y J, Yang M H. Motion Blur Kernel Estimation via Deep Learning. IEEE Transactions on Image Processing, 2018, 27(1): 194-205 doi: 10.1109/TIP.2017.2753658
    [9] Xu L, Ren J S, Liu C, Jia J Y. Deep convolutional neural network for image deconvolution. In: Proceedings of the 2014 International Conference on Neural Information Processing Systems. Montreal, Canada: Springer, 2014. 1790−1798
    [10] Tao X, Gao H Y, Shen X Y, Wang J, Jia J Y. Scale-Recurrent network for deep image deblurring. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, Utah, USA: IEEE, 2018. 8174−8182
    [11] Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D. Generative Adversarial Networks. Advances in Neural Information Processing Systems, 2014, (3): 2672-2680
    [12] 吴梦婷, 李伟红, 龚卫国. 双框架卷积神经网络用于运动模糊图像盲复原. 计算机辅助设计与图形学学报, 2018, 30(12): 2327-2334

    Wu M T, Li W H, Gong W G. Two-frame Convolutional Neural Network for Blind Motion Image Deblurring. Journal of Computer-Aided Design & Computer Graphics, 2018, 30(12): 2327-2334
    [13] Kupyn O, Budzan V, Mykhailych M, Mishkin D. DeblurGAN: Blind motion deblurring using conditional adversarial networks. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, Utah, USA: IEEE, 2018. 1−9
    [14] Tomasi C, Manduchi R. Bilateral filtering for gray and color images. In: Proceedings of the 1998 International Conference on Computer Vision. Bombay, India: IEEE, 1998. 839−846
    [15] Xu L, Lu C W, Xu Y, Jia J Y. Image Smoothing via L0 Gradient Minimization. ACM Transactions on Graphics, 2011, 30(6): 174.1-174.11
    [16] Wan R J, Shi B X, Duan L Y, Tan A H. CRRN: Multi-Scale guided concurrent reflection removal network. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, Utah, USA: IEEE, 2018. 4777−4785
    [17] 林景栋, 吴欣怡, 柴毅, 尹宏鹏. 卷积神经网络结构优化综述. 自动化学报, 2020, 46(1): 24-37

    Lin J D, Wu X Y, Chai Y, Yin H P. Structure Optimization of Convolutional Neural Networks: A Survey. Acta Automatica Sinica, 2020, 46(1): 24-37
    [18] Deng J, Dong W, Socher R, Li L J, Li K, Li F F. ImageNet: A large-scale hierarchical image database. In: Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recogniltion. Florida, USA: IEEE, 2009: 1−8
    [19] Mao X D, Li Q, Xie H, Raymond Y K L, Wang Z, Stephen P S. Least squares generative adversarial networks. In: Proceedings of the 2017 International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 2813−2821
    [20] PyTorch. PyTorch Documentation [Online], available: https://pytorch.org/docs/stable/index.html, September 1, 2019
    [21] Schuler C J, Hirsch M, Harmeling S, Scholkopf B. Learning to Deblur. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2016, 38(7): 1439-1451
    [22] 孙季丰, 朱雅婷, 王恺. 基于DeblurGAN和低秩分解的去运动模糊. 华南理工大学学报(自然科学版), 2020, 48(1): 32-41

    Sun J F, Zhu Y T, Wang K. Motion Deblurring Based on DeblurGAN and Low Rank Decomposition. Journal of South China University of Technology (Natural Science Edition), 2020, 48(1): 32-41
    [23] Kupyn O, Martyniuk T, Wu J R, Wang Z Y. DeblurGAN-v2: Deblurring (orders-of-magnitude) faster and better. In: Proceedings of the 2019 International Conference on Computer Vision. Seoul, Korea: IEEE, 2019. 8878−8887
  • 加载中
图(15) / 表(4)
计量
  • 文章访问数:  830
  • HTML全文浏览量:  214
  • PDF下载量:  182
  • 被引次数: 0
出版历程
  • 收稿日期:  2019-09-12
  • 录用日期:  2020-05-07
  • 网络出版日期:  2021-09-07
  • 刊出日期:  2021-11-18

目录

    /

    返回文章
    返回