2.845

2023影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于全局局部协同的非均匀图像去雾方法

罗小同 杨汶锦 曲延云 谢源

罗小同, 杨汶锦, 曲延云, 谢源. 基于全局局部协同的非均匀图像去雾方法. 自动化学报, 2024, 50(7): 1333−1344 doi: 10.16383/j.aas.c230567
引用本文: 罗小同, 杨汶锦, 曲延云, 谢源. 基于全局局部协同的非均匀图像去雾方法. 自动化学报, 2024, 50(7): 1333−1344 doi: 10.16383/j.aas.c230567
Luo Xiao-Tong, Yang Wen-Jin, Qu Yan-Yun, Xie Yuan. Dehazeformer: Nonhomogeneous image dehazing with collaborative global-local network. Acta Automatica Sinica, 2024, 50(7): 1333−1344 doi: 10.16383/j.aas.c230567
Citation: Luo Xiao-Tong, Yang Wen-Jin, Qu Yan-Yun, Xie Yuan. Dehazeformer: Nonhomogeneous image dehazing with collaborative global-local network. Acta Automatica Sinica, 2024, 50(7): 1333−1344 doi: 10.16383/j.aas.c230567

基于全局局部协同的非均匀图像去雾方法

doi: 10.16383/j.aas.c230567
基金项目: 国家自然科学基金 (62176224, 62222602) 资助
详细信息
    作者简介:

    罗小同:厦门大学信息学院博士研究生. 主要研究方向为计算机视觉与图像处理. E-mail: xiaotluo@stu.xmu.edu.cn

    杨汶锦:厦门大学信息学院硕士研究生. 主要研究方向为计算机视觉与图像处理. E-mail: wjyang6@stu.xmu.edu.cn

    曲延云:厦门大学信息学院教授. 主要研究方向为模式识别, 计算机视觉和机器学习. 本文通信作者. E-mail: yyqu@xmu.edu.cn

    谢源:华东师范大学计算机科学与技术学院教授. 主要研究方向为模式识别, 计算机视觉和机器学习. E-mail: yxie@cs.ecnu.edu.cn

Dehazeformer: Nonhomogeneous Image Dehazing With Collaborative Global-local Network

Funds: Supported by National Natural Science Foundation of China (62176224, 62222602)
More Information
    Author Bio:

    LUO Xiao-Tong Ph.D. candidate at the School of Informatics, Xiamen University. Her research interest covers computer vision and image processing

    YANG Wen-Jin Master student at the School of Informatics, Xiamen University. His research interest covers computer vision and image processing

    QU Yan-Yun Professor at the School of Informatics, Xiamen University. Her research interest covers pattern recognition, computer vision, and machine learning. Corresponding author of this paper

    XIE Yuan Professor at the School of Computer Science and Technology, East China Normal University. His research interest covers pattern recognition, computer vision, and machine learning

  • 摘要: 近年来, 基于卷积神经网络(Convolutional neural network, CNN) 的图像去雾方法在合成数据集上取得了显著的进展, 但由于真实场景中存在雾分布不均的问题, 卷积运算的局部感受野难以有效捕获到上下文指导信息, 从而导致全局结构信息丢失. 因此, 真实场景下的图像去雾任务面临着巨大的挑战. 考虑到Transformer具有捕获长距离语义信息依赖关系的优势, 有利于引导全局结构信息重建. 然而, 标准Transformer结构的高计算复杂度阻碍了其在图像恢复中的应用. 针对上述提到的问题, 提出一个由Transformer和卷积神经网络组成的双分支协同非均匀图像去雾网络Dehazeformer. Transformer分支用于提取全局结构信息, 同时设计稀疏自注意力模块(Sparse self-attention modules, SSM) 以降低计算复杂度. 卷积神经网络分支用于获取局部信息, 从而恢复纹理细节. 在真实非均匀有雾场景下的实验结果表明, 该方法不管是在客观评价还是在主观视觉效果方面均达到优异的性能.
  • 图  1  大气散射模型示意图

    Fig.  1  Schematic diagram of atmospheric scattering model

    图  2  基于全局局部协同的非均匀图像去雾网络Dehazeformer结构示意图

    Fig.  2  Schematic diagram of architecture of collaborative global-local network Dehazeformer for nonhomogeneous image dehazing

    图  3  标准Transformer层和本文提出的混合Transformer层结构对比

    Fig.  3  Architecture comparison of the standard Transformer layer and the proposed hybrid Transformer layer

    图  4  纹理重建分支的特征融合模块

    Fig.  4  Feature fusion module of texture reconstruction branch

    图  5  特征增强模块及注意力模块

    Fig.  5  Feature enhancement module and attention modules

    图  6  在NH-HAZE21测试集上消融实验的视觉比较

    Fig.  6  Visual comparison of ablation experiments on the NH-HAZE21 test dataset

    图  7  在NH-HAZE20测试集上与主流去雾方法的视觉比较

    Fig.  7  Visual comparison with mainstream dehazing methods on the NH-HAZE20 test dataset

    图  8  在NH-HAZE21测试集上与主流去雾方法的视觉比较

    Fig.  8  Visual comparison with mainstream dehazing methods on the NH-HAZE21 test dataset

    表  1  在NH-HAZE21测试集上的消融实验定量比较

    Table  1  Quantitative comparison of ablation experiments on the NH-HAZE21 test dataset

    方法 HTL SSM SRB TRB PSNR (dB) $ \uparrow $ SSIM$ \uparrow $ LPIPS$ \downarrow $
    Baseline × × $\checkmark$ × 21.24 0.8339 0.1849
    Baseline+HTL $\checkmark$ × $\checkmark$ × 21.88 0.8428 0.1766
    Baseline+SSM × $\checkmark$ $\checkmark$ × 21.54 0.8361 0.1837
    SRB $\checkmark$ $\checkmark$ $\checkmark$ × 22.08 0.8458 0.1752
    TRB × × × $\checkmark$ 21.62 0.8566 0.1740
    Ours $\checkmark$ $\checkmark$ $\checkmark$ $\checkmark$ 22.44 0.8631 0.1597
    下载: 导出CSV

    表  2  混合Transformer层与Twin Transformer层在NH-HAZE21测试集上的消融实验定量比较

    Table  2  Quantitative comparison of ablation experiments between hybrid Transformer layer and Twin Transformer layer on the NH-HAZE21 test dataset

    方法 PSNR (dB) $ \uparrow $ SSIM$ \uparrow $ LPIPS$ \downarrow $
    本文方法+位置嵌入编码 22.33 0.8611 0.1613
    Twin Transformer层 22.29 0.8611 0.1613
    混合Transformer层 22.44 0.8631 0.1597
    下载: 导出CSV

    表  3  在NH-HAZE20和NH-HAZE21测试集上与主流去雾方法的定量比较 (注: — 表示该方法未提供源码)

    Table  3  Quantitative comparison with mainstream dehazing methods on the NH-HAZE20 and NH-HAZE21 test datasets (Note: — indicates that the method does not provide source code)

    方法NH-HAZE20NH-HAZE21NH-HAZE平均值
    PSNR (dB)$ \uparrow $SSIM$ \uparrow $LPIPS$ \downarrow $PSNR (dB)$ \uparrow $SSIM$ \uparrow $LPIPS$ \downarrow $PSNR (dB)$ \uparrow $SSIM$ \uparrow $LPIPS$ \downarrow $
    DCP[25]11.640.45330.536511.570.62780.448611.610.56740.4926
    CAP[26]11.540.41880.572411.560.58480.486511.550.50180.5295
    AOD-Net[27]13.440.413015.200.64130.310314.320.5272
    GridDehaze[28]17.630.66680.304620.080.81340.233218.860.74010.2689
    FFA-Net[4]17.440.65430.334020.510.81390.231518.980.73410.2828
    MSBDN[2]19.010.70330.285820.890.82070.239319.950.76200.2626
    KDDN[3]17.250.66020.312120.640.81560.217018.950.73790.2646
    AECR[5]18.580.65750.280920.810.82690.186519.700.74220.2337
    MPSHAN[15]18.130.641018.970.781018.550.7110
    TransWeather[29]19.600.69900.269921.720.83680.197220.660.76790.2336
    Res2Net+RCAN[8]21.440.704021.660.843021.550.7735
    DB-CGAN[16]18.290.633019.330.791018.810.7120
    FADehaze[30]17.440.630020.500.840018.970.7350
    BiN-Flow[31]18.630.6340
    PFONet[32]20.090.6583
    SDD[33]22.150.8350
    TUSR-Net[34]21.960.7254
    ITBdehaze[11]21.440.710021.670.838021.560.7740
    本文方法22.160.73450.250122.440.86310.159722.300.79880.2049
    下载: 导出CSV

    表  4  与NTIRE 2021非均匀图像去雾挑战赛优胜方案的定量比较

    Table  4  Quantitative comparison with winning schemes of the nonhomogeneous image dehazing challenge in NTIRE 2021

    方法 PSNR (dB) $\uparrow $ SSIM $\uparrow $
    DWT dehaze 21.99 0.8560
    Mac dehaze 21.66 0.8430
    Bilibili AI & FDU 21.24 0.7882
    VIP UNIST 21.17 0.8360
    Buaa colab 20.13 0.8034
    本文方法 22.44 0.8631
    下载: 导出CSV
  • [1] Nayar S K, Narasimhan S G. Vision in bad weather. In: Proceedings of the Seventh IEEE/CVF International Conference on Computer Vision (ICCV). Kerkyra, Greece: IEEE, 1999. 820–827
    [2] Dong H, Pan J S, Xiang L, Hu Z, Zhang X Y, Wang F, et al. Multi-scale boosted dehazing network with dense feature fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 2154–2164
    [3] Hong M, Xie Y, Li C H, Qu Y Y. Distilling image dehazing with heterogeneous task imitation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 3459–3468
    [4] Qin X, Wang Z L, Bai Y C, Xie X D, Jia H Z. FFA-Net: Feature fusion attention network for single image dehazing. In: Proceedings of the Thirty-fourth AAAI Conference on Artificial Intelligence. New York, USA: AAAI Press, 2020. 11908–11915
    [5] Wu H Y, Qu Y Y, Lin S H, Zhou J, Qiao R Z, Zhang Z Z, et al. Contrastive learning for compact single image dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2021. 10546–10555
    [6] Liu J, Wu H Y, Xie Y, Qu Y Y, Ma L Z. Trident dehazing network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Seattle, USA: IEEE, 2020. 1732–1741
    [7] Wu H Y, Liu J, Xie Y, Qu Y Y, Ma L Z. Knowledge transfer dehazing network for nonhomogeneous dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Seattle, USA: IEEE, 2020. 1975–1983
    [8] Yu Y K, Liu H, Fu M H, Chen J, Wang X Y, Wang K Y. A two-branch neural network for non-homogeneous dehazing via ensemble learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Nashville, USA: IEEE, 2021. 193–202
    [9] Fu M H, Liu H, Yu Y K, Chen J, Wang K Y. DW-GAN: A discrete wavelet transform GAN for nonhomogeneous dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Nashville, USA: IEEE, 2021. 203–212
    [10] Guo C L, Yan Q X, Anwar S, Cong R M, Ren W Q, Li C Y. Image dehazing Transformer with transmission-aware 3D position embedding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). New Orleans, USA: IEEE, 2022. 5802–5810
    [11] Liu Y Y, Liu H, Li L Y, Wu Z J, Chen J. A data-centric solution to nonhomogeneous dehazing via vision Transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Vancouver, Canada: IEEE, 2023. 1406–1415
    [12] Das S D, Dutta S. Fast deep multi-patch hierarchical network for nonhomogeneous image dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Seattle, USA: IEEE, 2020. 1994–2001
    [13] Metwaly K, Li X L, Guo T T, Monga V. Nonlocal channel attention for nonhomogeneous image dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Seattle, USA: IEEE, 2020. 1842–1851
    [14] Ye T, Chen E K, Huang X R, Chen P. Efficient re-parameterization residual attention network for nonhomogeneous image dehazing. arXiv preprint arXiv: 2109.05479, 2021.
    [15] 杨坤, 张娟, 方志军. 基于多补丁和多尺度层级聚合网络的快速非均匀图像去雾. 计算机科学, 2021, 48(11): 250−257 doi: 10.11896/jsjkx.200900058

    Yang Kun, Zhang Juan, Fang Zhi-Jun. Multi-patch and multi-scale hierarchical aggregation network for fast nonhomogeneous image dehazing. Computer Science, 2021, 48(11): 250−257 doi: 10.11896/jsjkx.200900058
    [16] 朱利安, 张鸿. 基于双分支条件生成对抗网络的非均匀图像去雾. 计算机应用, 2023, 43(2): 567−574

    Zhu Li-An, Zhang Hong. Nonhomogeneous image dehazing based on dual-branch conditional generative adversarial network. Journal of Computer Applications, 2023, 43(2): 567−574
    [17] Liu Z, Lin Y T, Cao Y, Hu H, Wei Y X, Zhang Z, et al. Swin Transformer: Hierarchical vision Transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, Canada: IEEE, 2021. 9992–10002
    [18] Zhang Y L, Li K P, Li K, Wang L C, Zhong B N, Fu Y. Image super-resolution using very deep residual channel attention networks. In: Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: Springer, 2018. 294–310
    [19] Han K, Xiao A, Wu E H, Guo J Y, Xu C J, Wang Y H. Transformer in Transformer. arXiv preprint arXiv: 2103.00112, 2021.
    [20] Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X H, Unterthiner T, et al. An image is worth 16×16 words: Transformers for image recognition at scale. arXiv preprint arXiv: 2010.11929, 2021.
    [21] Guo R H, Niu D T, Qu L, Li Z B. SOTR: Segmenting objects with Transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, Canada: IEEE, 2021. 7137–7146
    [22] Zhao G X, Lin J Y, Zhang Z Y, Ren X C, Su Q, Sun X. Explicit sparse Transformer: Concentrated attention through explicit selection. arXiv preprint arXiv: 1912.11637, 2019.
    [23] Johnson J, Alahi A, Li F F. Perceptual losses for real-time style transfer and super-resolution. In: Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, 2016. 694–711
    [24] Lin T Y, Dollár P, Girshick R, He K M, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 936–944
    [25] He K M, Sun J, Tang X O. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341−2353 doi: 10.1109/TPAMI.2010.168
    [26] Zhu Q S, Mai J M, Shao L. A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing, 2015, 24(11): 3522−3533 doi: 10.1109/TIP.2015.2446191
    [27] Li B Y, Peng X L, Wang Z Y, Xu J Z, Feng D. AOD-Net: All-in-one dehazing network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017. 4780–4788
    [28] Liu X H, Ma Y R, Shi Z H, Chen J. GridDehazeNet: Attention-based multi-scale network for image dehazing. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, South Korea: IEEE, 2019. 7313–7322
    [29] Valanarasu J M J, Yasarla R, Patel V M. TransWeather: Transformer-based restoration of images degraded by adverse weather conditions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE, 2022. 2343–2353
    [30] 吴正平, 程洁莹, 雷帮军, 赵俊臣. 基于特征注意力的快速非均匀雾图像去雾算法. 国外电子测量技术, 2023, 42(9): 9−18

    Wu Zheng-Ping, Cheng Jie-Ying, Lei Bang-Jun, Zhao Jun-Chen. Fast nonhomogeneous image dehazing algorithm based on feature attention. Foreign Electronic Measurement Technology, 2023, 42(9): 9−18
    [31] Wu Y Q, Tao D P, Zhan Y B, Zhang C Y. BiN-Flow: Bidirectional normalizing flow for robust image dehazing. IEEE Transactions on Image Processing, 2022, 31: 6635−6648 doi: 10.1109/TIP.2022.3214093
    [32] Li S S, Zhou Y, Ren W Q, Xiang W. PFONet: A progressive feedback optimization network for lightweight single image dehazing. IEEE Transactions on Image Processing, 2023, 32: 6558−6569 doi: 10.1109/TIP.2023.3333564
    [33] Kim G, Kwon J. Self-parameter distillation dehazing. IEEE Transactions on Image Processing, 2022, 32: 631−642
    [34] Song X B, Zhou D F, Li W, Dai Y C, Shen Z L, Zhang L J, et al. TUSR-Net: Triple unfolding single image dehazing with self-regularization and dual feature to pixel attention. IEEE Transactions on Image Processing, 2023, 32: 1231−1244 doi: 10.1109/TIP.2023.3234701
    [35] Chen T Y, Fu J H, Jiang W T, Gao C, Liu S. SRKTDN: Applying super resolution method to dehazing task. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Nashville, USA: IEEE, 2021. 487–496
    [36] Jo E, Sim J Y. Multi-scale selective residual learning for non-homogeneous dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Nashville, USA: IEEE, 2021. 507–515
  • 加载中
图(8) / 表(4)
计量
  • 文章访问数:  306
  • HTML全文浏览量:  67
  • PDF下载量:  81
  • 被引次数: 0
出版历程
  • 收稿日期:  2023-09-12
  • 录用日期:  2024-03-17
  • 网络出版日期:  2024-06-25
  • 刊出日期:  2024-07-23

目录

    /

    返回文章
    返回