2.624

2020影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于语义引导特征聚合的显著性目标检测网络

王正文 宋慧慧 樊佳庆 刘青山

王正文, 宋慧慧, 樊佳庆, 刘青山. 基于语义引导特征聚合的显著性目标检测网络. 自动化学报, 2021, 48(x): 1001−1010 doi: 10.16383/j.aas.c210425
引用本文: 王正文, 宋慧慧, 樊佳庆, 刘青山. 基于语义引导特征聚合的显著性目标检测网络. 自动化学报, 2021, 48(x): 1001−1010 doi: 10.16383/j.aas.c210425
Wang Zheng-Wen, Song Hui-Hui, Fan Jia-Qing, Liu Qing-Shan. Semantic guided feature aggregation network for salient object detection. Acta Automatica Sinica, 2021, 48(x): 1001−1010 doi: 10.16383/j.aas.c210425
Citation: Wang Zheng-Wen, Song Hui-Hui, Fan Jia-Qing, Liu Qing-Shan. Semantic guided feature aggregation network for salient object detection. Acta Automatica Sinica, 2021, 48(x): 1001−1010 doi: 10.16383/j.aas.c210425

基于语义引导特征聚合的显著性目标检测网络

doi: 10.16383/j.aas.c210425
基金项目: 国家自然科学基金项目(61872189, 61532009), 江苏省自然科学基金项目(BK20191397), 江苏省六大人才高峰项目(XYDXX-015)
详细信息
    作者简介:

    王正文:南京信息工程大学自动化学院硕士研究生.主要研究方向为显著性目标检测, 深度学习. E-mail: 20191223064@nuist.edu.cn

    宋慧慧:南京信息工程大学自动化学院教授.主要研究方向为视频目标分割, 图像超分. 本文通讯作者. E-mail: songhuihui@nuist.edu.cn

    樊佳庆:南京航空航天大学计算机科学与技术学院博士研究生.主要研究方向为视频目标分割. E-mail: jqfan@nuaa.edu.cn

    刘青山:南京信息工程大学自动化学院教授.主要研究方向为视频内容分析与理解. E-mail: qsliu@nuist.edu.cn

Semantic Guided Feature Aggregation Network for Salient Object Detection

Funds: Supported by National Natural Science Foundation of China (61872189, 61532009), the Natural Science Foundation of Jiangsu Province (BK20191397), Six talent peaks project in Jiangsu Province under Grant nos. XYDXX-015
More Information
    Author Bio:

    WANG Zheng-Wen Master student at the School of Automation, Nanjing University of information science and technology. His research interest covers salient object detection and deep learning

    SONG Hui-Hui Professor at the School of Automation, Nanjing University of information science and technology. Her research interest covers video object segmentation and image super-resolution. Corresponding author of this paper

    FAN Jia-Qing PhD candidate with the College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China. His main research interest is video object segmentation

    LIU Qing-Shan Professor at the School of Automation, Nanjing University of information science and technology. His research interest covers video content analysis and understanding

  • 摘要: 在显著性目标检测网络的设计中, U型结构使用广泛. 但是U型结构显著性检测方法中普遍存在空间位置细节丢失和边缘难以细化的问题, 针对这些问题, 本文提出了一个基于语义信息引导特征聚合的网络, 通过高效的特征聚合来获得精细的显著性图. 网络由3部分组成, 分别是混合注意力模块, 增大感受野模块以及多层次聚合模块. 首先, 利用增大感受野模块处理特征提取网络提取出的低层特征, 使其在保留原有边缘细节的同时增大感受野, 以获得更加丰富的空间上下文信息. 然后, 利用混合注意力模块处理特征提取网络的最后一层特征, 以增强其表征力, 并作为解码过程中的语义指导, 不断指导特征聚合. 最后, 多层次聚合模块对来自不同层次的特征进行有效聚合, 得到最终精细的显著性图. 本文在6个基准数据集上进行了广泛的实验, 结果证明了该方法能够有效的定位显著特征, 并且对边缘细节的细化也很有效.
  • 图  1  网络结构图

    Fig.  1  Network structure diagram

    图  2  混合注意力模块

    Fig.  2  Mixing attention module

    图  3  增大感受野模块

    Fig.  3  Enlarged receptive field module

    图  4  多层次融合模块

    Fig.  4  Multi-level aggregation module

    图  5  不同算法的PR曲线示意图

    Fig.  5  Comparison of PR curves of different methods

    图  6  不同算法的显著性图

    Fig.  6  Saliency maps of different methods

    表  1  不同方法的${F_\beta }$指标结果比较

    Table  1  Comparison of ${F_\beta }$ values of different models

    OursPAGR[19]RAS[21]DGRL[36]CPD[23]MLMS[37]PoolNet[38]AFNet[39]BASNet[40]U2Net[15]ITSD[41]
    ECSSD0.9510.9240.9210.9210.9360.9300.9440.9350.9420.9510.947
    DUT-O0.8270.7710.7860.7740.7940.7930.8080.7970.8050.8230.824
    PASCAL-S0.8730.8470.8370.8440.8660.8580.8690.8680.8540.8590.871
    HKU-IS0.9370.9190.9130.9100.9240.9220.9330.9230.9280.9350.934
    DUTS-TE0.8880.8550.8310.8280.8640.8540.8800.8620.8600.8730.883
    SOD0.8730.8380.8100.8430.8500.8620.8670.8510.8610.880
    注: ${F_\beta }$值越大越好, 加粗字体为最优结果, 加下划线字体为次优结果. DUT-O代表DUT-OMRON
    下载: 导出CSV

    表  2  不同方法的MAE指标结果比较

    Table  2  Comparison of MAE values of different models

    OursPAGR[19]RAS[21]DGRL[36]CPD[23]MLMS[37]PoolNet[38]AFNet[39]BASNet[40]U2Net[15]ITSD[41]
    ECSSD0.0340.0640.0560.0430.0400.0380.0390.0420.0370.0340.035
    DUT-O0.0580.0710.0620.0620.0560.0600.0560.0570.0560.0540.061
    PASCAL-S0.0650.0890.1040.0720.0740.0690.0750.0690.0760.0740.072
    HKU-IS0.0320.0470.0450.0360.0330.0340.0330.0360.0320.0310.031
    DUTS-TE0.0420.0530.0600.0490.0430.0450.0400.0460.0470.0440.041
    SOD0.0930.1450.1240.1030.1120.1060.1000.1140.1080.095
    注: MAE值越小越好.
    下载: 导出CSV

    表  3  不同方法的${S_m}$指标结果比较

    Table  3  Comparison of ${S_m}$ values of different models

    OursPAGR[19]RAS[21]DGRL[36]CPD[23]MLMS[37]PoolNet[38]AFNet[39]BASNet[40]U2Net[15]ITSD[41]
    ECSSD0.9320.8890.8930.9060.9150.9110.9210.9140.9160.9280.925
    DUT-O0.8470.7750.8140.8100.8180.8170.8360.8260.8360.8470.840
    PASCAL-S0.8650.7490.7950.8690.8440.8490.8450.8500.8380.8440.859
    HKU-IS0.9300.8870.8870.8970.9040.9010.9170.9050.9090.9160.917
    DUTS-TE0.8730.8380.8390.8420.8670.8560.8830.8660.8530.8610.872
    SOD0.8080.7200.7640.7710.7710.7800.7950.7720.7860.809
    注: ${S_{\text{m}}}$值越大越好.
    下载: 导出CSV

    表  4  消融实验.

    Table  4  Ablation experiment

    MAMERFMMLAMMAE/${F_\beta }$
    0.049/0.935
    0.045/0.937
    0.042/0.942
    0.039/0.944
    0.034/0.951
    注:加粗字体为最优结果.
    下载: 导出CSV

    表  5  ERFM模块中不同空洞率配置的对比实验

    Table  5  Comparative experiment of different dilation rate configuration in ERFM

    空洞率的不同设置组合MAE/${F_\beta }$
    (1, 3, 5), (1, 3, 5), (1, 3, 5), (1, 3, 5)0.039/0.946
    (1, 3, 5), (1, 3, 5), (3, 5, 7), (1, 3, 5)0.037/0.948
    (1, 3, 5), (4, 6, 8), (3, 5, 7), (1, 3, 5)0.036/0.950
    (5, 8, 11), (4, 6, 8), (3, 5, 7), (1, 3, 5)0.034/0.951
    下载: 导出CSV

    表  6  关于MLAM模块中双分支的消融实验, MLAM↑指自下而上分支, MLAM↓指自上而下分支

    Table  6  Ablation experiment of MLAM. MLAM↑ represents a down to top branch and MLAM↓ represents a top to down branch

    MLAM↑MLAM↓MAE/${F_\beta }$
    0.041/0.940
    0.040/0.946
    0.034/0.951
    下载: 导出CSV

    表  7  MAM中注意力模块位置关系的消融实验

    Table  7  Ablation experiment on the position relationship of attention module in MAM

    注意力模块之间的位置关系MAE/${F_\beta }$
    Channel first0.036/0.947
    Spatial first0.038/0.944
    Parallel0.034/0.951
    下载: 导出CSV
  • [1] Donoser M, Urschler M, Hirzer M, and Bischof H. Saliency driven total variation segmentation. In: Proceedings of the 2009 IEEE International Conference on Computer Vision. Kyoto, Japan: IEEE, 2009. 817−824
    [2] Wei J, Wang S, and Huang Q. F3Net: Fusion, Feedback and Focus for Salient Object Detection. In: Proceedings of the 2020 AAAI Conference on Artificial Intelligence. New York, USA: Spring, 2020. 12321−12328
    [3] 李东民, 李静, 梁大川, 王超. 基于多尺度先验深度特征的多目标显著性检测方法. 自动化学报, 2019, 45(11): 2058-2070

    Li Dong-Min, Li Jing, Liang Da-Chuan, Wang Chao. Multiple Salient Objects Detection Using Multi-scale Prior and Deep Features. Acta Automatica Sinica, 2019, 45(11): 2058-2070
    [4] 徐威, 唐振民. 利用层次先验估计的显著性目标检测. 自动化学报, 2015, 41(4): 799-812

    Xu Wei, Tang Zhen-Min. Exploiting Hierarchical Prior Estimation for Salient Object Detection. Acta Automatica Sinica, 2015, 41(4): 799-812
    [5] 杨赛, 赵春霞, 徐威. 一种基于词袋模型的新的显著性目标检测方法. 自动化学报, 2016, 42(8): 1259-1273

    Yang Sai, Zhao Chun-Xia, Xu Wei. A novel salient object detection method using bag-of-features. Acta Automatica Sinica, 2016, 42(8): 1259-1273
    [6] Hong S, You T, Kwak S, and Han B. Online tracking by learning discriminative saliency map with convolutional neural network. In: Proceedings of the 32th International Conference on Machine Learning. Miami, USA: IMLS, 2015. 597−606
    [7] Ren Z, Gao S, Chia L, and Tsang I W. Region-based saliency detection and its application in object recognition. In: IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(5): 769−779
    [8] Wang X, You S, Li X, and Ma H. Weakly-supervised semantic segmentation by iteratively mining common object features. In: Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 1354−1362
    [9] Goferman S, Zelnik-Manor L, and Tal A. Context-aware saliency detection. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(10): 1915−1926
    [10] Yan Q, Xu L, Shi J, and Jia J. Hierarchical saliency detection. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, America: IEEE, 2013. 1155−1162
    [11] Hou Q, Cheng M M, Hu X, Borji A, Tu Z and Torr P. Deeply supervised salient object detection with short connections. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Hawaii, USA: IEEE, 2017. 5300−5309
    [12] Luo Z, Mishra A, Achkar A, Eichel J, and Jodoin PM. Non-local deep features for salient object detection. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Hawaii, USA: IEEE, 2017. 6609−6617
    [13] Zhang P, Wang D, Lu H, Wang H, and Ruan X. Amulet: Aggregating multi-level convolutional features for salient object detection. In: Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017. 202−211
    [14] Li X, Zhao L M, Wei L, Yang M, Wu F, Zhuang Y T, Ling H B, and Wang J D. DeepSaliency: Multi-task deep neural network model for salient object detection. In: IEEE Transactions on Image Processing, 2016, 25(8): 3919−3930
    [15] Qin X, Zhang Z, Huang C, Dehghan M, and Jagersand M. U2-Net: Going deeper with nested U-structure for salient object detection. In: Pattern Recognition, 2020, 106: 107404
    [16] Pang Y, Zhao X, Zhang L, and Lu H. Multi-scale Interactive Network for Salient Object Detection. In: Proceedings of the 2020 IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020. 9413−9422
    [17] Wei J, Wang S, Wu Z, Su C, Huang Q, and Tian Q. Label Decoupling Framework for Salient Object Detection. In: Proceedings of the 2020 IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020. 13025−13034
    [18] Borji A, Cheng M M, Jiang H and Li J. Salient object detection: A benchmark. In: IEEE Transactions on Image Processing, 2015, 24(12): 5706−5722
    [19] Zhang X N, Wang T T, Qi J Q, Lu H C, and Wang G. Progressive attention guided recurrent network for salient object detection. In: Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 714−722
    [20] Zhao T, Wu X Q. Pyramid feature attention network for saliency detection. In: Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Los Angeles, USA: IEEE, 2019. 3085−3094
    [21] Chen S, Tan X, Wang B, and Hu X. Reverse attention for salient object detection. In: Proceedings of the 2018 IEEE Europeon Conference on Computer Vision. Munich, Germany: IEEE, 2018. 234−250
    [22] Wang W, Zhao S, Shen J, Hoi S C, and Borji A. Salient object detection with pyramid attention and salient edges. In: Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Los Angeles, USA: IEEE, 2019. 1448−1457
    [23] Wu Z, Su L, and Huang Q. Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Los Angeles, USA: IEEE, 2019. 3907−3916
    [24] Deng Z J, Hu X W, Zhu L, Xu X M, Qin J, Han G Q, and Heng P. R3net: Recurrent residual refinement network for saliency detection. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. Stockholm, Sweden: AAAI Press, 2018. 684−690
    [25] Wang B, Chen Q, Zhou M, Zhang Z, Jin X, and Gai K. Progressive feature polishing network for salient object detection. In: Proceedings of the 2020 AAAI Conference on Artificial Intelligence. New York, USA: Spring, 2020
    [26] Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, and Lu H. Dual Attention Network for Scene Segmentation. In: Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Los Angeles, USA: IEEE, 2019. 3146−3154
    [27] Zhao H S, Shi J P, Qi X J, Wang X G, and Jia J Y. Pyramid Scene Parsing Network. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Hawaii, USA: IEEE, 2017. 6230−6239
    [28] Wang L J, Lu H C, Wang Y F, Feng M Y, Wang D, Yin B C, and Ruan X. Learning to detect salient objects with image-level supervision. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Hawaii, USA: IEEE, 2017. 3796−3805
    [29] Yang C, Zhang L, Lu H C, Ruan X, and Yang M. Saliency detection via graph-based manifold ranking. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, USA: IEEE, 2013. 3166−-3173
    [30] Li X H, Lu H C, Zhang L, Ruan X, and Yang M. Saliency detection via dense and sparse reconstruction. In: Proceedings of the 2013 IEEE International Conference on Computer Vision. Sydney, Australia: IEEE, 2013. 2976−2983
    [31] Li G B, Yu Y Z. Visual saliency based on multiscale deep features. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015. 5455−5463
    [32] Li Y, Hou X, Koch C, Rehg J M, and Yuille A L. The secrets of salient object segmentation. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014. 280v287
    [33] Movahedi V, Elder J H. Design and perceptual validation of performance measures for salient object segmentation. In: Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. San Francisco, USA: IEEE, 2010. 49−56
    [34] Kingma D P, Ba J. Adam: A method for stochastic optimization. ArXiv preprint ArXiv: 1412.6980, 2014
    [35] Fan D P, Cheng M M, Liu Y, Li T, and Borji A. Structure-measure: A new way to evaluate foreground maps. In: Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017. 4548−4557
    [36] Wang T T, Zhang L, Wang S, Lu H C, Yang G, Ruan Y and Borji A. Detect globally, refine locally: A novel approach to saliency detection. In: Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 3127−3135
    [37] Wu R, Feng M, Guan W, Wang D, Lu H, and Ding E. A mutual learning method for salient object detection with intertwined multi-supervision. In: Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Los Angeles, USA: IEEE, 2019. 8150−8159
    [38] Liu JJ, Hou Q, Cheng MM, Feng J, and Jiang J. PoolNet: A Simple Pooling-Based Design for Real-Time Salient Object Detection. In: Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Los Angeles, USA: IEEE, 2019. 3917−3926
    [39] Feng M Y, Lu H C, Ding E. Attentive feedback network for boundary-aware salient object detection. In: Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Los Angeles, USA: IEEE, 2019. 1623−1632
    [40] Qin X B, Zhang Z C, Huang C Y, Gao C, Dehghan M, and Jagersand M. BASNet: Boundary-Aware Salient Object Detection. In: Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Los Angeles, USA: IEEE, 2019
    [41] Zhou H, Xie X, Lai J H, Chen Z, and Yang L. Interactive Two-Stream Decoder for Accurate and Fast Saliency Detection. In: Proceedings of the 2020 IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020. 9141−9150
  • 加载中
计量
  • 文章访问数:  351
  • HTML全文浏览量:  190
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-05-18
  • 录用日期:  2020-10-18
  • 网络出版日期:  2021-11-15

目录

    /

    返回文章
    返回