2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于跨连接LeNet-5网络的面部表情识别

李勇 林小竹 蒋梦莹

李勇, 林小竹, 蒋梦莹. 基于跨连接LeNet-5网络的面部表情识别. 自动化学报, 2018, 44(1): 176-182. doi: 10.16383/j.aas.2018.c160835
引用本文: 李勇, 林小竹, 蒋梦莹. 基于跨连接LeNet-5网络的面部表情识别. 自动化学报, 2018, 44(1): 176-182. doi: 10.16383/j.aas.2018.c160835
LI Yong, LIN Xiao-Zhu, JIANG Meng-Ying. Facial Expression Recognition with Cross-connect LeNet-5 Network. ACTA AUTOMATICA SINICA, 2018, 44(1): 176-182. doi: 10.16383/j.aas.2018.c160835
Citation: LI Yong, LIN Xiao-Zhu, JIANG Meng-Ying. Facial Expression Recognition with Cross-connect LeNet-5 Network. ACTA AUTOMATICA SINICA, 2018, 44(1): 176-182. doi: 10.16383/j.aas.2018.c160835

基于跨连接LeNet-5网络的面部表情识别

doi: 10.16383/j.aas.2018.c160835
基金项目: 

国家自然科学基金 60772168

详细信息
    作者简介:

    李勇 北京化工大学硕士研究生.主要研究方向为图像处理与模式识别, 深度学习.E-mail:15117965051@163.com

    蒋梦莹 北京化工大学硕士研究生.主要研究方向为图像处理与模式识别, 深度学习.E-mail:18810493772@163.com

    通讯作者:

    林小竹 北京石油化工学院教授.主要研究方向为图像处理与模式识别, 深度学习, 信号与系统.本文通信作者.E-mail:linzhu1964@163.com

Facial Expression Recognition with Cross-connect LeNet-5 Network

Funds: 

National Natural Science Foundation of China 60772168

More Information
    Author Bio:

    Master student at the College of Information Science and Technology, Beijing University of Chemical Technology. His research interest covers image processing, pattern recognition, and deep learning

    Master student at the College of Information Science and Technology, Beijing University of Chemical Technology. Her research interest covers image processing, pattern recognition, and deep learning

    Corresponding author: LIN Xiao-Zhu Professor at the School of Information Engineering, Beijing Institute of Petrochemical Technology. His research interest covers image processing and pattern recognition, deep learning, and signals and systems. Corresponding author of this paper
  • 摘要: 为避免人为因素对表情特征提取产生的影响,本文选择卷积神经网络进行人脸表情识别的研究.相较于传统的表情识别方法需要进行复杂的人工特征提取,卷积神经网络可以省略人为提取特征的过程.经典的LeNet-5卷积神经网络在手写数字库上取得了很好的识别效果,但在表情识别中识别率不高.本文提出了一种改进的LeNet-5卷积神经网络来进行面部表情识别,将网络结构中提取的低层次特征与高层次特征相结合构造分类器,该方法在JAFFE表情公开库和CK+数据库上取得了较好的结果.
    1)  本文责任编委 胡清华
  • 图  1  LeNet-5结构图

    Fig.  1  The LeNet-5 convolutional neural network

    图  2  改进的LeNet-5卷积神经网络

    Fig.  2  Improved LeNet-5 convolutional neural network

    图  3  JAFFE表情库7种表情示例图像

    Fig.  3  7 kinds of facial expression image in JAFFE expression dataset

    图  4  CK+表情库7种表情示例图像

    Fig.  4  7 kinds of facial expression image in the CK+ expression dataset

    表  1  LeNet-5网络Layer 2与Layer 3之间的连接方式

    Table  1  Connection between LeNet-5 network0s Layer 2 and Layer 3

    12345678910111213141516
    1
    2
    3
    4
    5
    6
    下载: 导出CSV

    表  2  卷积网络参数

    Table  2  Convolutional network parameters

    输入输入尺寸卷积核大小池化区域步长输出尺寸
    Input32 × 325 × 5128 × 28
    Layer 16 @ 28 × 282 × 226@14 × 14
    Layer 26 @ 14 × 145 × 5110 × 10
    Layer 316 @ 10 × 102 × 2216 @ 5 × 5
    Layer 416 @ 5 × 55 × 51120@1 × 1
    Layer 5120 @ 1 × 11 × 84
    Layer 61 × 1 6601 × 7
    Output1 × 7
    下载: 导出CSV

    表  3  JAFFE表情库不同表情的分类正确率(%)

    Table  3  Classification accuracy of different expressions in JAFFE expression dataset (%)

    生气厌恶害怕高兴中性悲伤惊讶整体
    测试集11008010010010090.9188.8994.37
    测试集2100909081.8210010010092.96
    测试集310010081.8290.9110010010095.77
    整体10089.6690.6390.6310096.7796.5594.37
    下载: 导出CSV

    表  4  CK+数据库不同表情的分类正确率(%)

    Table  4  Classification accuracy of different expressions in CK+ dataset (%)

    生气厌恶害怕高兴中性悲伤惊讶整体
    测试集188.8994.448092.8670.839693.9488.89
    测试集270.3777.788096.30688496.9782.32
    测试集377.7885.7184.62100647293.9483.33
    测试集462.9694.298889.29608087.8880.81
    测试集581.4885.717292.866479.1710083.33
    整体76.3087.5980.9294.2665.3782.2394.5583.74
    下载: 导出CSV

    表  5  网络是否跨连接正确率对比(%)

    Table  5  Classification accuracy of the network whether cross connection or not (%)

    方法参数量JAFFE表情库中平均正确率CK+数据库中平均正确率
    LeNet-514 44462.4432.32
    本文方法25 47694.3783.74
    下载: 导出CSV

    表  6  不同方法在JAFFE上的对比(%)

    Table  6  The comparison of different methods on JAFFE (%)

    来源方法正确率
    Kumbhar等[28]*Image feature60 ~ 70
    Praseeda等[5]*SVM86.9
    本文算法跨连的LeNet-594.37
      *数据来源于文献[15]
    下载: 导出CSV
  • [1] Pantic M, Rothkrantz L J M. Expert system for automatic analysis of facial expressions. Image and Vision Computing, 2000, 18(11):881-905 doi: 10.1016/S0262-8856(00)00034-2
    [2] Ekman P, Friesen W V. Facial Action Coding System:A Technique for the Measurement of Facial Movement. Palo Alto, CA:Consulting Psychologists Press, 1978. https://www.researchgate.net/publication/239537771_Facial_action_coding_system_A_technique_for_the_measurement_of_facial_movement
    [3] Lucey P, Cohn J F, Kanade T, Saragih J, Ambadar Z, Matthews I. The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). San Francisco, CA, USA: IEEE, 2010. 94-101 http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=5543262
    [4] Lanitis A, Taylor C J, Cootes T F. Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(7):743-756 doi: 10.1109/34.598231
    [5] Praseeda Lekshmi V, Sasikumar M. Analysis of facial expression using Gabor and SVM. International Journal of Recent Trends in Engineering, 2009, 1(2):47-50 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.381.5275
    [6] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, USA: NIPS, 2012. 1097-1105 http://dl.acm.org/citation.cfm?id=2999257
    [7] Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks. Science, 2006, 313(5786):504-507 doi: 10.1126/science.1127647
    [8] 余凯, 贾磊, 陈雨强, 徐伟.深度学习的昨天、今天和明天.计算机研究与发展, 2013, 50(9):1799-1804 doi: 10.7544/issn1000-1239.2013.20131180

    Yu Kai, Jia Lei, Chen Yu-Qiang, Xu Wei. Deep learning:yesterday, today, and tomorrow. Journal of Computer Research and Development, 2013, 50(9):1799-1804 doi: 10.7544/issn1000-1239.2013.20131180
    [9] 王梦来, 李想, 陈奇, 李澜博, 赵衍运.基于CNN的监控视频事件检测.自动化学报, 2016, 42(6):892-903 http://www.aas.net.cn/CN/abstract/abstract18880.shtml

    Wang Meng-Lai, Li Xiang, Chen Qi, Li Lan-Bo, Zhao Yan-Yun. Surveillance event detection based on CNN. Acta Automatica Sinica, 2016, 42(6):892-903 http://www.aas.net.cn/CN/abstract/abstract18880.shtml
    [10] 奚雪峰, 周国栋.面向自然语言处理的深度学习研究.自动化学报, 2016, 42(10):1445-1465 http://www.aas.net.cn/CN/abstract/abstract18934.shtml

    Xi Xue-Feng, Zhou Guo-Dong. A survey on deep learning for natural language processing. Acta Automatica Sinica, 2016, 42(10):1445-1465 http://www.aas.net.cn/CN/abstract/abstract18934.shtml
    [11] 张晖, 苏红, 张学良, 高光来.基于卷积神经网络的鲁棒性基音检测方法.自动化学报, 2016, 42(6):959-964 http://www.aas.net.cn/CN/abstract/abstract18887.shtml

    Zhang Hui, Su Hong, Zhang Xue-Liang, Gao Guang-Lai. Convolutional neural network for robust pitch determination. Acta Automatica Sinica, 2016, 42(6):959-964 http://www.aas.net.cn/CN/abstract/abstract18887.shtml
    [12] 随婷婷, 王晓峰.一种基于CLMF的深度卷积神经网络模型.自动化学报, 2016, 42(6):875-882 http://www.aas.net.cn/CN/abstract/abstract18878.shtml

    Sui Ting-Ting, Wang Xiao-Feng. Convolutional neural networks with candidate location and multi-feature fusion. Acta Automatica Sinica, 2016, 42(6):875-882 http://www.aas.net.cn/CN/abstract/abstract18878.shtml
    [13] 王伟凝, 王励, 赵明权, 蔡成加, 师婷婷, 徐向民.基于并行深度卷积神经网络的图像美感分类.自动化学报, 2016, 42(6):904-914 http://www.aas.net.cn/CN/abstract/abstract18881.shtml

    Wang Wei-Ning, Wang Li, Zhao Ming-Quan, Cai Cheng-Jia, Shi Ting-Ting, Xu Xiang-Min. Image aesthetic classification using parallel deep convolutional neural networks. Acta Automatica Sinica, 2016, 42(6):904-914 http://www.aas.net.cn/CN/abstract/abstract18881.shtml
    [14] 常亮, 邓小明, 周明全, 武仲科, 袁野, 杨硕, 王宏安.图像理解中的卷积神经网络.自动化学报, 2016, 42(9):1300-1312 http://www.aas.net.cn/CN/abstract/abstract18919.shtml

    Chang Liang, Deng Xiao-Ming, Zhou Ming-Quan, Wu Zhong-Ke, Yuan Ye, Yang Shuo, Wang Hong-An. Convolutional neural networks in image understanding. Acta Automatica Sinica, 2016, 42(9):1300-1312 http://www.aas.net.cn/CN/abstract/abstract18919.shtml
    [15] 孙晓, 潘汀, 任福继.基于ROI-KNN卷积神经网络的面部表情识别.自动化学报, 2016, 42(6):883-891 http://www.aas.net.cn/CN/abstract/abstract18879.shtml

    Sun Xiao, Pan Ting, Ren Fu-Ji. Facial expression recognition using ROI-KNN deep convolutional neural networks. Acta Automatica Sinica, 2016, 42(6):883-891 http://www.aas.net.cn/CN/abstract/abstract18879.shtml
    [16] Hubel D H, Wiesel T N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. The Journal of Physiology, 1962, 160(1):106-154 doi: 10.1113/jphysiol.1962.sp006837
    [17] Fukushima K, Miyake S, Ito T. Neocognitron:a neural network model for a mechanism of visual pattern recognition. IEEE Transactions on Systems, Man, and Cybernetics, 1983, SMC-13(5):826-834 doi: 10.1109/TSMC.1983.6313076
    [18] Le Cun Y, Boser B, Denker J S, Howard R E, Habbard W, Jackel L D, Henderson D. Handwritten digit recognition with a back-propagation network. Advances in Neural Information Processing Systems 2. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1989. 396-404
    [19] Le Cun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11):2278-2324 doi: 10.1109/5.726791
    [20] Bengio Y. Learning deep architectures for AI. Foundations and Trends® in Machine Learning, 2009, 2(1):1-127 doi: 10.1561/2200000006
    [21] Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS) 2010. Sardinia, Italy: Chia Laguna Resort, 2010. 249-256
    [22] Ziegel R. Modern Applied Statistics with S-plus (3rd edition), by Venables W N and Ripley B D, New York: Springer-Verlag, 1999, Technometrics, 2001, 43(2): 249
    [23] Srivastava R K, Greff K, Schmidhuber J. Highway networks. Computer Science, arXiv: 1505. 00387, 2015.
    [24] Romero A, Ballas N, Kahou S E, Chassang A, Gatta C, Bengio Y. FitNets: hints for thin deep nets. Computer Science, arXiv: 1412. 6550, 2014.
    [25] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. arXiv: 1512. 03385, 2016. 770-778
    [26] Sun Y, Wang X G, Tang X O. Deep learning face representation from predicting 10, 000 classes. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Columbus, OH, USA: IEEE, 2014. 1891-1898 https://www.computer.org/csdl/proceedings/cvpr/2014/5118/00/5118b891-abs.html
    [27] 张婷, 李玉鑑, 胡海鹤, 张亚红.基于跨连卷积神经网络的性别分类模型.自动化学报, 2016, 42(6):858-865 http://www.aas.net.cn/CN/abstract/abstract18876.shtml

    Zhang Ting, Li Yu-Jian, Hu Hai-He, Zhang Ya-Hong. A gender classification model based on cross-connected convolutional neural networks. Acta Automatica Sinica, 2016, 42(6):858-865 http://www.aas.net.cn/CN/abstract/abstract18876.shtml
    [28] Kumbhar M, Jadhav A, Patil M. Facial expression recognition based on image feature. International Journal of Computer and Communication Engineering, 2012, 1(2):117-119 https://www.researchgate.net/publication/250922449_Facial_Expression_Recognition_Based_on_Image_Feature
  • 加载中
图(4) / 表(6)
计量
  • 文章访问数:  4024
  • HTML全文浏览量:  650
  • PDF下载量:  1438
  • 被引次数: 0
出版历程
  • 收稿日期:  2016-12-23
  • 录用日期:  2017-05-04
  • 刊出日期:  2018-01-20

目录

    /

    返回文章
    返回