2.845

2023影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于并行深度卷积神经网络的图像美感分类

王伟凝 王励 赵明权 蔡成加 师婷婷 徐向民

王伟凝, 王励, 赵明权, 蔡成加, 师婷婷, 徐向民. 基于并行深度卷积神经网络的图像美感分类. 自动化学报, 2016, 42(6): 904-914. doi: 10.16383/j.aas.2016.c150718
引用本文: 王伟凝, 王励, 赵明权, 蔡成加, 师婷婷, 徐向民. 基于并行深度卷积神经网络的图像美感分类. 自动化学报, 2016, 42(6): 904-914. doi: 10.16383/j.aas.2016.c150718
WANG Wei-Ning, WANG Li, ZHAO Ming-Quan, CAI Cheng-Jia, SHI Ting-Ting, XU Xiang-Min. Image Aesthetic Classification Using Parallel Deep Convolutional Neural Networks. ACTA AUTOMATICA SINICA, 2016, 42(6): 904-914. doi: 10.16383/j.aas.2016.c150718
Citation: WANG Wei-Ning, WANG Li, ZHAO Ming-Quan, CAI Cheng-Jia, SHI Ting-Ting, XU Xiang-Min. Image Aesthetic Classification Using Parallel Deep Convolutional Neural Networks. ACTA AUTOMATICA SINICA, 2016, 42(6): 904-914. doi: 10.16383/j.aas.2016.c150718

基于并行深度卷积神经网络的图像美感分类

doi: 10.16383/j.aas.2016.c150718
基金项目: 

广东省前沿与关键技术创新专项资金(重大科技专项) 2014B010111003

广东省前沿与关键技术创新专项资金(重大科技专项) 2014B010111006

国家自然科学基金 61401161

国家自然科学基金 61171142

广东省自然科学基金 2015A030313212

详细信息
    作者简介:

    王伟凝 华南理工大学电子与信息学院副教授. 2005年获得华南理工大学通信与信息系统专业博士学位. 主要研究方向为图像处理与模式识别, 计算机视觉, 机器学习

    王励 南理工大学电子与信息学院硕士研究生. 主要研究方向为图像处理, 机器学习与计算机视觉

    赵明权 华南理工大学电子与信息学院硕士研究生. 主要研究方向为计算机视觉, 机器学习

    蔡成加 华南理工大学电子与信息学院硕士研究生. 主要研究方向为图像处理, 机器学习和计算机视觉

    师婷婷 华南理工大学电子与信息学院硕士研究生. 2015年获郑州大学电子信息科学与技术学士学位. 主要研究方向为图像处理, 计算机视觉与模式识别

    通讯作者:

    徐向民 华南理工大学电子与信息学院教授. 2001年获华南理工大学电子与信息学院博士学位. 主要研究方向为图像/视频处理, 人机交互, 计算机视觉与机器学习. 本文通信作者. E-mail: xmxu@scut.edu.cn

  • 中图分类号: 

Image Aesthetic Classification Using Parallel Deep Convolutional Neural Networks

Funds: 

Guandong Frontier and Key Technological Innovation Special Funds (Grant Scienti¯c and Technological Project) 2014B010111003

Guandong Frontier and Key Technological Innovation Special Funds (Grant Scienti¯c and Technological Project) 2014B010111006

National Natural Science Foundation of China 61401161

National Natural Science Foundation of China 61171142

Natural Science Foundation of Guandong Province 2015A030313212

More Information
    Author Bio:

    WANG Wei-Ning Associate professor at the School of Electronic and Information Engineering, South China University of Technology. She received her Ph. D. degree from South China University of Technology in 2005. Her research interest covers image processing, pattern recognition, computer vision, and machine learning

    WANG Li Master student at the School of Electronic and Information Engineering, South China University of Technology. Her research interest covers image processing, machine learning, and computer vision

    ZHAO Ming-Quan Master student at the School of Electronic and Information Engineering, South China University of Technology. His research interest covers computer vision and machine learning

    CAI Cheng-Jia Master student at the School of Electronic and Information Engineering, South China University of Technology. His research interest covers image processing, machine learning, and computer vision

    SHI Ting-Ting Master student at the School of Electronic and Information Engineering, South China University of Technology. She received her bachelor degree from Zhengzhou University in 2015. Her research interest covers image processing, computer vision, and pattern recognition

    Corresponding author: XU Xiang-Min Professor at the School of Electronic and Information Engineering, South China University of Technology. He re- ceived his Ph. D. degree from the School of Electronic and Information Engineering, South China University of Tech- nology in 2001. His research interest covers image/video processing, human-computer interaction, computer vision, and machine learning. Corresponding author of this paper. E-mail: xmxu@scut.edu.cn
  • 摘要: 随着计算机和社交网络的飞速发展, 图像美感的自动评价产生了越来越大的需求并受到了广泛关注. 由于图像美感评价的主观性和复杂性, 传统的手工特征和局部特征方法难以全面表征图像的美感特点, 并准确量化或建模. 本文提出一种并行深度卷积神经网络的图像美感分类方法, 从同一图像的不同角度出发, 利用深度学习网络自动完成特征学习, 得到更为全面的图像美感特征描述; 然后利用支持向量机训练特征并建立分类器, 实现图像美感分类. 通过在两个主流的图像美感数据库上的实验显示, 本文方法与目前已有的其他算法对比, 获得了更好的分类准确率.
  • 图  1  本文方法的整体框架

    Fig.  1  The overall framework of the method in this paper

    图  2  单路卷积神经网络

    Fig.  2  Single column convolutional neural networks

    图  3  影响图像美感的主要因素示例

    Fig.  3  The main factors affecting aesthetics of images

    图  4  卷积神经网络的不同图像输入形式示例

    Fig.  4  Examples of different types of input images of convolutional neural networks

    图  5  数据库中高美感和低美感图像示例

    Fig.  5  Examples of high aesthetic images and low aesthetic images of datasets

    表  1  不同结构单路卷积神经网络的分类准确率

    Table  1  Classification accuracy of single column convolutional neural networks with different structures

    全连接FcFcFcFcFcFc分类准确率
    层设置4096204810245122562(%)
    Arch183.70
    Arch283.73
    Arch383.21
    Arch483.28
    下载: 导出CSV

    表  2  不同输入的单路卷积神经网络的分类准确率

    Table  2  Classification accuracy of single column convolutional neural networks with different inputs

    输入方式分类准确率(%)
    Normal83.28
    Resize80.28
    H70.03
    S75.90
    V82.99
    Daubechies81.60
    下载: 导出CSV

    表  3  各种特征组合方式的分类准确率

    Table  3  Classification accuracy of various

    输入组合NormalResizeHSVDaubechies特征维数分类准确率
    (%)
    176883.93
    225683.28
    351283.66
    451284.18
    551285.00
    676885.17
    776885.33
    876885.83
    9102485.41
    10128085.94
    下载: 导出CSV

    表  4  AVA1数据库的实验结果及与现有方法的对比

    Table  4  The experimental results of the AVA1 datasets and comparison with existing methods

    方法类型图像美感分类方法分类准确率(%)
    手工特征Datta 等[6]68.67*
    Ke 等[8]71.06*
    局部特征Marchesotti 等[11]68.55*
    深度学习方法DCNN Aesth SP[16]83.52
    本文方法85.00
    * 此行数据引用自文献[16] 的结果.
    下载: 导出CSV

    表  5  AVA2数据库的实验结果及与现有方法的对比

    Table  5  The experimental results of the AVA2 datasets and comparison with existing methods

    图像美感分类方法分类准确率(%)
    RDCNN semantic[10]75.42
    本文方法77.03
    下载: 导出CSV

    表  6  CUHKPQ 各类别图库和总图库的实验结果及现有方法的对比

    Table  6  The experimental results of the CUHKPQ datasets and comparison with existing methods

    特征类型场景类别
    AnimalArchitectureHumanLandscapeNightPlantStaticOverall
    手工特征All features in [8]*0.77510.85260.79080.81700.73210.80930.78290.7944
    All features in [9]0.89370.92750.97400.94680.84630.91820.90690.9209
    局部特征Semantic features[12]0.86230.86440.93130.84160.87420.86850.89640.8787
    Semantic features + handcrafted features[12]0.90330.87550.94720.88530.90520.92320.90940.9093
    深度学习方法DCNN Aesth SP[16]-------0.9193
    本文方法0.93820.91130.96970.91000.91660.94100.91590.9395
    * 此行数据引用自文献[12]的结果.
    下载: 导出CSV
  • [1] 王伟凝, 蚁静缄, 贺前华. 可计算图像美学研究进展. 中国图象图形学报, 2012,17(8): 893-901

    Wang Wei-Ning, Yi Jing-Jian, He Qian-Hua. Review for computational image aesthetics. Journal of Image and Graphics, 2012,17(8): 893-901
    [2] 王伟凝, 刘剑聪, 徐向民, 姜怡孜, 王励. 基于构图规则的图像美学优化. 华南理工大学学报(自然科学版), 2015,43(5): 51-58

    Wang Wei-Ning, Liu Jian-Cong, Xu Xiang-Min, Jiang Yi-Zi, Wang Li. Aesthetic enhancement of images based on photography composition guidelines. Journal of South China University of Technology (Natural Science Edition), 2015,43(5): 51-58
    [3] 王伟凝, 蚁静缄, 徐向民, 王励. 可计算的图像美学分类与评估. 计算机辅助设计与图形学学报, 2014,26(7): 1075-1083

    Wang Wei-Ning, Yi Jing-Jian, Xu Xiang-Min, Wang Li. Computational aesthetics of image classification and evaluation. Journal of Computer-Aided Design & Computer Graphics, 2014,26(7): 1075-1083
    [4] Wang W N, Cai D, Wang L, Huang Q H, Xu X M, Li X L. Synthesized computational aesthetic evaluation of photos. Neurocomputing, 2016,172: 244-252
    [5] Tong H H, Li M J, Zhang H J, He J R, Zhang C S. Classification of digital photos taken by photographers or home users. In: Proceedings of the 5th Pacific Rim Conference on Multimedia. Tokyo, Japan: Springer, 2004. 198-205
    [6] Datta R, Joshi D, Li J, Wang J Z. Studying aesthetics in photographic images using a computational approach. In: Proceedings of the 9th European Conference on Computer Vision. Graz, Austria: Springer, 2006. 288-301
    [7] Wang W N, Zhao W J, Cai C J, Huang J X, Xu X M, Li L. An efficient image aesthetic analysis system using Hadoop. Signal Processing: Image Communication, 2015,39: 499-508
    [8] Ke Y, Tang X O, Jing F. The design of high-level features for photo quality assessment. In: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE, 2006. 419-426
    [9] Tang X O, Luo W, Wang X G. Content-based photo quality assessment. IEEE Transactions on Multimedia, 2013,15(8): 1930-1943
    [10] Lu X, Lin Z, Jin H L, Yang J C, Wang J Z. Rating image aesthetics using deep learning. IEEE Transactions on Multimedia, 2015,17(11): 2021-2034
    [11] Marchesotti L, Perronnin F, Larlus D, Csurka G. Assessing the aesthetic quality of photographs using generic image descriptors. In: Proceedings of the 2011 IEEE International Conference on Computer Vision. Barcelona, Spain: IEEE, 2011. 1784-1791
    [12] Guo L H, Xiong Y C, Huang Q H, Li X L. Image esthetic assessment using both hand-crafting and semantic features. Neurocomputing, 2014,143: 14-26
    [13] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In: Proceedings of the 26th Annual Conference on Neural Information Processing Systems 2012. Lake Tahoe, USA: Curran Associates, Inc., 2012. 1097-1105
    [14] Sun Y, Wang X G, Tang X O. Deep learning face representation from predicting 10000 classes. In: Proceedings of the 27th IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014. 1891-1898
    [15] Lee H, Grosse R, Ranganath R, Ng A Y. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings of the 26th Annual International Conference on Machine Learning. Montreal, Canada: ACM, 2009. 609-616
    [16] Dong Z, Shen X, Li H Q, Tian X M. Photo quality assessment with DCNN that understands image well. In: Proceedings of the 21st International Conference on MultiMedia Modeling. Sydney, Australia: Springer International Publishing, 2015. 524-535
    [17] Dong Z, Tian X M. Multi-level photo quality assessment with multi-view features. Neurocomputing, 2015,168: 308-319
    [18] Deng J, Dong W, Socher R, Li L J, Li K, Li F F. ImageNet: a large-scale hierarchical image database. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA: IEEE, 2009. 248-255
    [19] Yin W, Mei T, Chen C W. Assessing photo quality with geo-context and crowdsourced photos. In: Proceedings of the 2012 IEEE Visual Communications and Image Processing. San Diego, USA: IEEE, 2012. 1-6
    [20] Murray N, Marchesotti L, Perronnin F. AVA: a large-scale database for aesthetic visual analysis. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA: IEEE, 2012. 2408-2415
    [21] Wang W N, Cai D, Xu X M, Liew A W C. Visual saliency detection based on region descriptors and prior knowledge. Signal Processing: Image Communication, 2014,29(3): 424-433
  • 加载中
图(5) / 表(6)
计量
  • 文章访问数:  3254
  • HTML全文浏览量:  1067
  • PDF下载量:  1138
  • 被引次数: 0
出版历程
  • 收稿日期:  2015-10-31
  • 录用日期:  2016-02-27
  • 刊出日期:  2016-06-20

目录

    /

    返回文章
    返回