2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于误差回传机制的多尺度去雾网络

杨爱萍 李晓晓 张腾飞 王朝臣 王建

杨爱萍, 李晓晓, 张腾飞, 王朝臣, 王建. 基于误差回传机制的多尺度去雾网络. 自动化学报, 2023, 49(9): 1857−1867 doi: 10.16383/j.aas.c210264
引用本文: 杨爱萍, 李晓晓, 张腾飞, 王朝臣, 王建. 基于误差回传机制的多尺度去雾网络. 自动化学报, 2023, 49(9): 1857−1867 doi: 10.16383/j.aas.c210264
Yang Ai-Ping, Li Xiao-Xiao, Zhang Teng-Fei, Wang Chao-Chen, Wang Jian. Multi-scale dehazing network based on error-backward mechanism. Acta Automatica Sinica, 2023, 49(9): 1857−1867 doi: 10.16383/j.aas.c210264
Citation: Yang Ai-Ping, Li Xiao-Xiao, Zhang Teng-Fei, Wang Chao-Chen, Wang Jian. Multi-scale dehazing network based on error-backward mechanism. Acta Automatica Sinica, 2023, 49(9): 1857−1867 doi: 10.16383/j.aas.c210264

基于误差回传机制的多尺度去雾网络

doi: 10.16383/j.aas.c210264
基金项目: 国家自然科学基金(62071323, 61771329, 61632018)资助
详细信息
    作者简介:

    杨爱萍:天津大学电气自动化与信息工程学院副教授. 主要研究方向为深度学习, 图像处理和计算机视觉. 本文通信作者. E-mail: yangaiping@tju.edu.cn

    李晓晓:天津大学电气自动化与信息工程学院硕士研究生. 主要研究方向为图像去雾, 深度学习. E-mail: leexx@tju.edu.cn

    张腾飞:天津大学电气自动化与信息工程学院硕士研究生. 主要研究方向为图像风格转换, 深度学习. E-mail: ztf951@gmail.com

    王朝臣:天津大学电气自动化与信息工程学院硕士研究生. 主要研究方向为图像去雨, 深度学习. E-mail: chen2019@tju.edu.cn

    王建:天津大学电气自动化与信息工程学院讲师. 主要研究方向为计算机视觉, 认知计算. E-mail: jianwang@tju.edu.cn

Multi-scale Dehazing Network Based on Error-backward Mechanism

Funds: Supported by National Natural Science Foundation of China (62071323, 61771329, 61632018)
More Information
    Author Bio:

    YANG Ai-Ping Associate professor at the School of Electrical and Infor-mation Engineering, Tianjin University. Her research interest covers deep learning, image processing, and computer vision. Corresponding author of this paper

    LI Xiao-Xiao Master student at the School of Electrical and Information Engineering, Tianjin Univer-sity. Her research interest covers image dehazing and deep learning

    ZHANG Teng-Fei Master student at the School of Electrical and Information Engineering, Tianjin University. His research interest covers image style transfer and deep learning

    WANG Chao-Chen Master student at the School of Electrical and Information Engineering, Tianjin University. His research interest covers image deraining and deep learning

    WANG Jian Lecturer at the Sch-ool of Electrical and Information Engineering, Tianjin University. His research interest covers computer vision and cognitive computing

  • 摘要: 针对现有图像去雾方法因空间上/下文信息丢失而无法准确估计大尺度目标特征, 导致图像结构被破坏或去雾不彻底等问题, 提出一种基于误差回传机制的多尺度去雾网络. 网络由误差回传多尺度去雾组(Error-backward multi-scale dehazing group, EMDG)、门控融合模块(Gated fusion module, GFM)和优化模块组成. 其中误差回传多尺度去雾组包括误差回传模块(Error-backward block, EB)和雾霾感知单元(Haze aware unit, HAU). 误差回传模块度量相邻尺度网络特征图之间的差异, 并将生成的差值图回传至上一尺度, 实现对结构信息和上/下文信息的有效复用; 雾霾感知单元是各尺度子网络的核心, 其由残差密集块(Residual dense block, RDB)和雾浓度自适应检测块(Haze density adaptive detection block, HDADB)组成, 可充分提取局部信息并能够根据雾浓度实现自适应去雾. 不同于已有融合方法直接堆叠各尺度特征, 提出的门控融合模块逐像素学习每个子网络特征图对应的最优权重, 有效避免了干扰信息对图像结构和细节信息的破坏. 再经优化模块, 可得到最终的无雾图像. 在合成数据集和真实数据集上的大量实验表明, 该方法优于目前的主流去雾方法, 尤其是对远景雾气去除效果更佳.
  • 图  1  直接融合策略和误差回传策略示意图

    Fig.  1  Illustration of direct-integration strategy and error-backward strategy for multi-scale network

    图  2  基于误差回传机制的多尺度去雾网络

    Fig.  2  Architecture of multi-scale dehazing network based on error-backward mechanism

    图  3  误差回传模块结构

    Fig.  3  Architecture of the error-backward block

    图  4  雾霾感知单元结构

    Fig.  4  The structure of the haze aware unit

    图  5  残差密集块结构

    Fig.  5  The structure of the residual dense block

    图  6  雾浓度自适应检测块结构

    Fig.  6  The structure of the haze density adaptive detection block

    图  7  与现有方法在SOTS测试集上去雾结果对比

    Fig.  7  Comparisons of dehazing results with state-of-the-art methods on SOTS

    图  8  HSTS测试集上与现有方法去雾结果对比

    Fig.  8  Comparisons of dehazing results with state-of-the-art methods on HSTS

    图  9  与现有方法在真实有雾图像上去雾结果对比

    Fig.  9  Comparisons of dehazing results with state-of-the-art methods on real hazy images

    图  10  部分区域色度偏暗的去雾图

    Fig.  10  Dehazed images with some darker areas

    表  1  SOTS室内测试集去雾结果的定量比较

    Table  1  Qualitative comparisons of dehazing results on SOTS indoor test-set

    方法DCPDehazeNetAODNetEPDNGCANet
    PSNR (dB)16.6221.1419.0625.0630.23
    SSIM0.81790.84720.85040.92320.9800
    方法GridDehazeNetPFDNYNetMSBDN本文方法
    PSNR (dB)32.1632.6819.0433.7933.83
    SSIM0.98360.97600.84650.98400.9834
    下载: 导出CSV

    表  2  SOTS室外测试集去雾结果的定量比较

    Table  2  Qualitative comparisons of dehazing results on SOTS outdoor test-set

    方法DCPDehazeNetMSCNNAODNet
    PSNR (dB)19.1322.4622.0620.29
    SSIM0.81480.85140.90780.8765
    方法EPDNGridDehazeNetYNet本文方法
    PSNR (dB)22.5730.8625.0231.10
    SSIM0.86300.98190.90120.9765
    下载: 导出CSV

    表  3  O-Haze数据集去雾结果定量比较

    Table  3  Qualitative comparisons of dehazing results on O-Haze data-set

    方法DCPMSCNNAODNetEPDNGCANetGridDehazeNet本文方法
    PSNR (dB)16.7817.2615.0316.0016.2818.9219.28
    SSIM0.65300.65010.53940.64130.64500.67210.6756
    下载: 导出CSV

    表  4  HSTS测试集去雾结果的定量比较

    Table  4  Qualitative comparisons of dehazing results on HSTS test-set

    方法DCPDehazeNetMSCNNAODNetEPDNYNetCCDID本文方法
    PSNR (dB)14.8424.4818.6420.5523.3818.3717.2230.07
    SSIM0.76090.91530.81680.89730.90590.47250.82180.9658
    下载: 导出CSV

    表  5  基于不同模块的网络性能比较

    Table  5  Comparisons of network performance based on different modules

    模块名称ABCDE
    5个RDB
    9个RDB
    GFM
    EB
    HDADB
    PSNR (dB)28.7929.5231.5332.4533.83
    下载: 导出CSV

    表  6  各方法平均运行时间对比

    Table  6  Average computing time comparison of various methods

    方法CPU/GPU时间 (s)
    DCPCPU25.08
    DehazeNetCPU2.56
    MSCNNCPU2.45
    AODNetGPU0.24
    GridDehazeNetGPU0.59
    FFANet[28]GPU1.23
    本文方法GPU0.73
    下载: 导出CSV
  • [1] Wonkyun K, Jong Y, Jechang J. Contrast enhancement using histogram equalization based on logarithmic mapping. Optical Engineering, 2012, 51(6), 067002: 1-11
    [2] Li H, Xie W H, Wang X G. GPU implementation of multi-scale retinex image enhancement algorithm. In: Proceedings of the IEEE/ACS International Conference of Computer Systems and Applications. Agadir, Morocco: IEEE, 2016. 1−5
    [3] Stark J A. Adaptive image contrast enhancement using generalizations of histogram equalization. IEEE Transactions on Image Processing, 2000, 9(5): 889-896 doi: 10.1109/83.841534
    [4] He K, Sun J, Tang X. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(12): 2341-2353
    [5] Tarel J P, Hautière N. Fast visibility restoration from a single color or gray level image. In: Proceedings of the IEEE International Conference on Computer Vision. Kyoto, Japan: IEEE, 2010. 2201−2208
    [6] Zhu Q S, Mai J M, Shao L. A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing, 2015. 24(11): 3522–3533 doi: 10.1109/TIP.2015.2446191
    [7] Berman D, Treibitz T, Avidan S. Non-local image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 1674−1682
    [8] 张小刚, 唐美玲, 陈华, 汤红忠. 一种结合双区域滤波和图像融合的单幅图像去雾算法. 自动化学报, 2014, 40(8): 1733-1739

    ZHANG Xiao-Gang, TANG Mei-Ling, CHEN Hua, TANG Hong-Zhong. A dehazing method in single image based on double-area filter and image fusion. Acta Automatica Sinica, 2014, 40(8): 1733-1739
    [9] 汪云飞, 冯国强, 刘华伟, 赵搏欣. 基于超像素的均值-均方差暗通道单幅图像去雾方法. 自动化学报, 2018, 44(3): 481-489

    WANG Yun-Fei, FENG Guo-Qiang, LIU Hua-Wei, ZHAO Bo-Xin. Superpixel-based mean and mean square deviation dark channel for single image fog removal. Acta Automatica Sinica, 2018, 44(3): 481-489
    [10] Cai B L, Xu X M, Jia K, Qing C M, Tao D C. DehazeNet: an end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 2016, 25(11): 5187-5198 doi: 10.1109/TIP.2016.2598681
    [11] Ren W Q, Liu S, Zhang H, Pan J S, Cao X C, Yang M H. Single image dehazing via multi-scale convolutional neural networks. In: Proceedings of the European Conference on Computer Vision. Amsterdam, Netherlands: 2016. 154−169
    [12] Zhang H, Patel V M. Densely connected pyramid dehazing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 3194−3203
    [13] Li B Y, Peng X L, Wang Z Y, Xu J Z, Feng D. AODNet: All-in-one dehazing network. In: Proceedings of the IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017. 4780−4788
    [14] Chen D D, He M M, Fan Q N, Liao J, Zhang L H, Hou D D, et al. Gated context aggregation network for image dehazing and deraining. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision. Hawaii, USA: IEEE, 2019. 1375−1383
    [15] Liu X H, Ma Y R, Shi Z H, Chen J. GridDehazeNet: Attention-based multi-scale network for image dehazing. In: Proceedings of the IEEE International Conference on Computer Vision. Seoul, South Korea: IEEE, 2019. 7314−7323
    [16] Dong J X, Pan J S. Physics-based feature dehazing networks. In: Proceedings of the Europeon Conference on Computer Vision. Glasgow, UK: 2020. 188−204
    [17] Yang H H, Yang C H H, Tsai Y C J. YNet: Multi-scale feature aggregation network with wavelet structure similarity loss function for single image dehazing. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona, Spain: IEEE, 2020. 2628−2632
    [18] Deng Q L, Huang Z L, Lin C W. HardGAN: A haze-aware representation distillation GAN for single image dehazing. In: Proceedings of the Europeon Conference on Computer Vision. Glasgow, UK: 2020. 722−738
    [19] Ronneberger O, Fischer P, Brox T. Unet: Convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: 2015. 234– 241
    [20] Johnson J, Alexandre A, Li F F. Perceptual losses for real-time style transfer and super-resolution. In: Proceedings of the Europeon Conference on Computer Vision. Amsterdam, Netherlands: 2016. 694−711
    [21] Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision. 2015. 211–252
    [22] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 3rd International Conference on Learning Representations. San Diego, USA: ICLR, 2015. 1−14
    [23] Li B Y, Ren W Q, Fu D P, Tao D C, Feng D, Zeng W J, et al. Benchmarking single image dehazing and beyond. IEEE Transactions on Image Processing, 2019. 28(1): 492-505 doi: 10.1109/TIP.2018.2867951
    [24] Ancuti C O, Ancuti C, Timofte R, Vleeschouwer C D. O-Haze: A dehazing benchmark with real hazy and haze-free outdoor images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City, USA: IEEE, 2018. 754−762
    [25] Qu Y Y, Chen Y Z, Huang J Y, Xie Y. Enhanced Pix2pix dehazing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019. 8160−8168
    [26] Dong H, Pan J S, Xiang L, Hu Z, Zhang X Y, Wang F, et al. Multi-scale boosted dehazing network with dense feature fusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Virtual Event: IEEE, 2020. 2154−2164
    [27] Dhara S K, Roy M, Sen D, Biswas P K. Color cast dependent image dehazing via adaptive airlight refinement and non-linear color balancing. IEEE Transactions on Circuits and Systems for Video Technology, 2020: 1-5
    [28] Qin X, Wang Z L, Bai Y C, Xie X D, Jia H Z. FFANet: Feature fusion attention network for single image dehazing. In: Proceedings of the Association for the Advance of Artificial Intelligence. New York, USA: AAAI Press, 2020. 11908−11915
  • 加载中
图(10) / 表(6)
计量
  • 文章访问数:  748
  • HTML全文浏览量:  332
  • PDF下载量:  161
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-03-31
  • 录用日期:  2021-09-17
  • 网络出版日期:  2021-11-05
  • 刊出日期:  2023-09-26

目录

    /

    返回文章
    返回