2.845

2023影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于条件随机森林的非约束环境自然笑脸检测

罗珍珍 陈靓影 刘乐元 张坤

罗珍珍, 陈靓影, 刘乐元, 张坤. 基于条件随机森林的非约束环境自然笑脸检测. 自动化学报, 2018, 44(4): 696-706. doi: 10.16383/j.aas.2017.c160439
引用本文: 罗珍珍, 陈靓影, 刘乐元, 张坤. 基于条件随机森林的非约束环境自然笑脸检测. 自动化学报, 2018, 44(4): 696-706. doi: 10.16383/j.aas.2017.c160439
LUO Zhen-Zhen, CHEN Jing-Ying, LIU Le-Yuan, ZHANG Kun. Conditional Random Forests for Spontaneous Smile Detection in Unconstrained Environment. ACTA AUTOMATICA SINICA, 2018, 44(4): 696-706. doi: 10.16383/j.aas.2017.c160439
Citation: LUO Zhen-Zhen, CHEN Jing-Ying, LIU Le-Yuan, ZHANG Kun. Conditional Random Forests for Spontaneous Smile Detection in Unconstrained Environment. ACTA AUTOMATICA SINICA, 2018, 44(4): 696-706. doi: 10.16383/j.aas.2017.c160439

基于条件随机森林的非约束环境自然笑脸检测

doi: 10.16383/j.aas.2017.c160439
基金项目: 

中央高校基本科研业务费 CCNU14A05 019

国家自然科学基金 41671377

中央高校基本科研业务费 CCNU16A02020

教育部中移动基金 MCM2013 0601

教育部人文社会科学研究基金 14YJAZH005

国家社科基金 16BSH107

中央高校基本科研业务费 CCNU14A05020

详细信息
    作者简介:

    罗珍珍, 华中师范大学国家数字化学习工程技术研究中心博士研究生.主要研究方向为计算机视觉, 模式识别, 图像处理.E-mail:andrealoves@163.com

    陈靓影, 华中师范大学国家数字化学习工程技术研究中心教授.主要研究方向为计算机视觉, 模式识别, 多模态人机交互.E-mail:chenjy@mail.ccnu.edu.cn

    张坤, 华中师范大学国家数字化学习工程技术研究中心讲师.主要研究方向为计算机视觉, 模式识别, 多模态人机交互.E-mail:zhk@mail.ccnu.edu.cn

    通讯作者:

    刘乐元, 华中师范大学国家数字化学习工程技术研究中心讲师.主要研究方向为计算机视觉, 模式识别, 多模态人机交互.本文通信作者.E-mail:lyliu@mail.ccnu.edu.cn

Conditional Random Forests for Spontaneous Smile Detection in Unconstrained Environment

Funds: 

the Colleges Basic Research and Operation of Ministry of Education CCNU14A05 019

Supported by National Natural Science Foundation of China 41671377

the Colleges Basic Research and Operation of Ministry of Education CCNU16A02020

Research Funds from Ministry of Education and China Mobile MCM2013 0601

Research Funds from the Humanities and Social Sciences Foundation of the Ministry of Education 14YJAZH005

National Social Sciences Foundation 16BSH107

the Colleges Basic Research and Operation of Ministry of Education CCNU14A05020

More Information
    Author Bio:

    Ph. D. candidate at the National Engineering Research Center for E-Learning, Central China Normal University. Her research interest covers computer vision, pattern recognition, and image processing

    Professor at the National Engineering Research Center for E-Learning, Central China Normal University. Her research interest covers computer vision, pattern recognition, and multimodal human-computer interaction

    Lecturer at the National Engineering Research Center for E-Learning, Central China Normal University. His research interest covers image processing, pattern recognition, and intelligent human-computer interaction

    Corresponding author: LIU Le-Yuan Lecturer at the National Engineering Research Center for E-Learning, Central China Normal University. His research interest covers computer vision, pattern recognition, and multimodal human-computer interaction. Corresponding author of this paper
  • 摘要: 为减少非约束环境下头部姿态多样性对笑脸检测带来的不利影响,提出一种基于条件随机森林(Conditional random forests,CRF)的笑脸检测方法.首先,以头部姿态作为隐含条件划分数据空间,构建基于条件随机森林的笑脸分类器;其次,以K-Means聚类方法确定条件随机森林分类器的分类边界;最后,分别从嘴巴区域和眉眼区域采集图像子块训练两组条件随机森林构成层级式结构进行笑脸检测.本文的笑脸检测方法在GENKI-4K、LFW和自备课堂场景(CCNU-Classroom)数据集上分别取得了91.14%,90.73%和85.17%的正确率,优于现有基于支持向量机、AdaBoost和随机森林的笑脸检测方法.
    1)  本文责任编委 黄庆明
  • 图  1  基于条件随机森林的笑脸检测示意图

    Fig.  1  Smile detection based on conditional random forests

    图  2  层级式笑脸检测流程图

    Fig.  2  The flowchart of the proposed smile detection method

    图  3  决策树的数量与笑脸分类准确率的关系

    Fig.  3  The accuracies for different numbers of trees in CRF

    图  4  本文方法的笑脸检测结果

    Fig.  4  The exemplar results of the proposed smile detection method

    表  1  本文方法与文献[15-16]在GENKI-4K数据集上的比较

    Table  1  The proposed approach compared with [15-16] on GENKI-4K dataset

    方法 特征 分类器 准确率(%)
    An等[16] LBP LDA 76.60
    An等[16] LBP SVM 84.20
    An等[16] HOG ELM 88.50
    Shan[15] LBP AdaBoost 86.43
    Shan[15] Gray AdaBoost 80.38
    Shan[15] Pixel Comparisons AdaBoost 89.70
    本文方法 LBP CRF 86.99
    本文方法 Gray CRF 88.36
    本文方法 LBP, Gray, Gabor CRF 91.14
    下载: 导出CSV

    表  2  头部姿态估计在LFW和CCNU-Classroom数据集上的准确率(%)

    Table  2  Accuracies of head pose estimation on LFW and CCNU-Classroom datasets (%)

    头部姿态 LFW CCNU-Classroom
    正脸 87.88 86.41
    微侧 80.00 81.60
    侧脸 83.73 83.33
    混合 82.72 83.41
    下载: 导出CSV

    表  3  不同笑脸检测算法在LFW和CCNU-Classroom数据集上的准确率(%)

    Table  3  Comparisons of accuracies of different smile detection algorithms on LFW and CCNU-Classroom datasets (%)

    LFW LFW CCNU-Classroom
    正脸 微侧 侧脸 混合 正脸 微侧 侧脸 混合
    本文 92.86 90.67 89.04 90.73 88.89 86.96 79.66 85.17
    SVM 85.63 77.00 81.85 83.25 77.56 74.51 68.53 73.52
    RF 78.00 77.14 85.99 81.74 78.89 79.85 59.17 72.38
    AdaBoost 75.00 72.35 68.54 71.96 70.00 65.56 61.24 66.27
    下载: 导出CSV

    表  4  不同图像子块采样方式在LFW数据集上的笑脸检测准确率(%)

    Table  4  Accuracies of smile detection with different image sub-regions on LFW dataset (%)

    头部姿态 整个人脸 嘴巴区域 眉眼区域 嘴巴十眉眼
    正脸 78.00 91.08 67.74 95.09
    微侧 75.50 88.50 64.50 90.05
    侧脸 72.08 86.86 62.08 86.86
    混合 74.79 88.71 64.59 90.47
    下载: 导出CSV

    表  5  不同嘴巴和眉眼区域定位方法的笑脸检测准确率(%)

    Table  5  Accuracies of smile detection using different approaches to locate eyes and mouth regions (%)

    方法 正脸 微侧 侧脸 混合
    几何关系粗略定位 95.09 90.05 86.86 90.47
    人脸特征点精确定位 95.79 91.00 88.74 91.37
    下载: 导出CSV

    表  6  使用不同决策边界方法对应的笑脸检测准确率(%)

    Table  6  Accuracies of smile detection using different decision boundary methods (%)

    LFW CCNU-Classroom
    头部姿态 K-Means 高斯 决策桩 K-Means 高斯 决策桩
    正脸 95.09 90.78 52.91 88.89 87.78 75.56
    微测 90.50 88.50 80.00 86.96 85.04 71.43
    侧脸 86.86 85.23 74.22 79.66 77.94 61.90
    混合 90.81 88.17 69.04 85.17 83.59 69.63
    下载: 导出CSV
  • [1] Sénéchal T, Turcot J, el Kaliouby R. Smile or smirk? Automatic detection of spontaneous asymmetric smiles to understand viewer experience. In: Proceedings of the 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG). Shanghai, China: IEEE, 2013. 1-8
    [2] Chen J Y, Luo N, Liu Y Y, Liu L Y, Zhang K, Kolodziej J. A hybrid intelligence-aided approach to affect-sensitive e-learning. Computing, 2016, 98(1-2):215-233 doi: 10.1007/s00607-014-0430-9
    [3] Shah R, Kwatra V. All smiles: automatic photo enhancement by facial expression analysis. In: Proceedings of the 9th European Conference on Visual Media Production (CVMP). London, UK: ACM, 2012. 1-10
    [4] Whitehill J, Littlewort G, Fasel I, Bartlett M, Movellan J. Toward practical smile detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(11):2106-2111 doi: 10.1109/TPAMI.2009.42
    [5] Sariyanidi E, Gunes H, Cavallaro A. Automatic analysis of facial affect:a survey of registration, representation, and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(6):1113-1133 doi: 10.1109/TPAMI.2014.2366127
    [6] 孙晓, 潘汀, 任福继.基于ROI-KNN卷积神经网络的面部表情识别.自动化学报, 2016, 42(6):883-891 http://www.aas.net.cn/CN/abstract/abstract18879.shtml

    Sun Xiao, Pan Ting, Ren Fu-Ji. Facial expression recognition using ROI-KNN deep convolutional neural networks. Acta Automatica Sinica, 2016, 42(6):883-891 http://www.aas.net.cn/CN/abstract/abstract18879.shtml
    [7] Tong Y, Chen J X, Ji Q. A unified probabilistic framework for spontaneous facial action modeling and understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(2):258-273 doi: 10.1109/TPAMI.2008.293
    [8] Vick S J, Waller B M, Parr L A, Pasqualini M C S, Bard K. A cross-species comparison of facial morphology and movement in humans and chimpanzees using the facial action coding system (FACS). Journal of Nonverbal Behavior, 2007, 31(1):1-20 doi: 10.1007/s10919-006-0017-z
    [9] Valstar M, Pantic M. Fully automatic recognition of the temporal phases of facial actions. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012, 42(1):28-43 doi: 10.1109/TSMCB.2011.2163710
    [10] 解仑, 卢亚楠, 姜波, 孙铁, 王志良.基于人脸运动单元及表情关系模型的自动表情识别.北京理工大学学报, 2016, 36(2):163-169 http://www.cnki.com.cn/Article/CJFDTotal-BJLG201602011.htm

    Xie Lun, Lu Ya-Nan, Jiang Bo, Sun Tie, Wang Zhi-Liang. Expression automatic recognition based on facial action units and expression relationship model. Transactions of Beijing Institute of Technology, 2016, 36(2):163-169 http://www.cnki.com.cn/Article/CJFDTotal-BJLG201602011.htm
    [11] 王磊, 邹北骥, 彭小宁.针对表情动作单元跟踪的隧道隐变量法.自动化学报, 2009, 35(2):198-201 http://www.aas.net.cn/CN/abstract/abstract18060.shtml

    Wang Lei, Zou Bei-Ji, Peng Xiao-Ning. Tunneled latent variables method for facial action unit tracking. Acta Automatica Sinica, 2009, 35(2):198-201 http://www.aas.net.cn/CN/abstract/abstract18060.shtml
    [12] Yang P, Liu Q S, Metaxas D N. Exploring facial expressions with compositional features. In: Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). San Francisco, CA, USA: IEEE, 2010. 2638-2644
    [13] Walecki R, Rudovic O, Pavlovic V, Pantic M. Variable-state latent conditional random fields for facial expression recognition and action unit detection. In: Proceedings of the 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG). Ljubljana, Slovenia: IEEE, 2015. 1-8
    [14] Shimada K, Matsukawa T, Noguchi Y, Kurita T. Appearance-based smile intensity estimation by cascaded support vector machines. In: Proceedings of the 2010 Revised Selected Papers, Part I Asian Conference on Computer Vision (ACCV). Queenstown, New Zealand: Springer, 2010. 277-286
    [15] Shan C F. Smile detection by boosting pixel differences. IEEE Transactions on Image Processing, 2012, 21(1):431-436 doi: 10.1109/TIP.2011.2161587
    [16] An L, Yang S F, Bhanu B. Efficient smile detection by extreme learning machine. Neurocomputing, 2015, 149:354-363 doi: 10.1016/j.neucom.2014.04.072
    [17] Huang G B, Zhou H M, Ding X J, Zhang R. Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012, 42(2):513-529 doi: 10.1109/TSMCB.2011.2168604
    [18] Gao Y, Liu H, Wu P P, Wang C. A new descriptor of gradients self-similarity for smile detection in unconstrained scenarios. Neurocomputing, 2016, 174:1077-1086 doi: 10.1016/j.neucom.2015.10.022
    [19] Liu H, Gao Y, Wu P. Smile detection in unconstrained scenarios using self-similarity of gradients features. In: Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP). Paris, France: IEEE, 2014. 1455-1459
    [20] El Meguid M K A, Levine M D. Fully automated recognition of spontaneous facial expressions in videos using random forest classifiers. IEEE Transactions on Affective Computing, 2014, 5(2):141-154 doi: 10.1109/TAFFC.2014.2317711
    [21] 刘帅师, 田彦涛, 万川.基于Gabor多方向特征融合与分块直方图的人脸表情识别方法.自动化学报, 2011, 37(12):1455-1463 http://www.aas.net.cn/CN/abstract/abstract17643.shtml

    Liu Shuai-Shi, Tian Yan-Tao, Wan Chuan. Facial expression recognition method based on gabor multi-orientation features fusion and block histogram. Acta Automatica Sinica, 2011, 37(12):1455-1463 http://www.aas.net.cn/CN/abstract/abstract17643.shtml
    [22] Dapogny A, Bailly K, Dubuisson S. Pairwise conditional random forests for facial expression recognition. In: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, USA: IEEE, 2015, 3783-3791
    [23] Yin L J, Wei X Z, Sun Y, Wang J, Rosato M J. A 3D facial expression database for facial behavior research. In: Proceedings of the 7th IEEE International Conference on Automatic Face and Gesture Recognition. Southampton, Britain: IEEE, 2006. 211-216
    [24] Huang G B, Mattar M, Berg T, Learned-Miller E. Labeled faces in the wild:a database for studying face recognition in unconstrained environments. Technical Report, University of Massachusetts, USA, 2007.
    [25] Breiman L. Random forests. Machine Learning, 2001, 45(1):5-32 doi: 10.1023/A:1010933404324
    [26] Liu Y Y, Chen J Y, Su Z M, Luo Z Z, Luo N, Liu L Y, Zhang K. Robust head pose estimation using Dirichlet-tree distribution enhanced random forests. Neurocomputing, 2015, 173:42-53 https://www.sciencedirect.com/science/article/pii/S0925231215010413
    [27] Sun M, Kohli P, Shotton J. Conditional regression forests for human pose estimation. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Providence, RI, USA: IEEE, 2012. 3394-3401
    [28] Dantone M, Gall J, Fanelli G, Van Gool L. Real-time facial feature detection using conditional regression forests. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Providence, RI, USA, 2012. 2578-2585
    [29] Du S Y, Zheng N N, You Q B, Wu Y, Yuan M J, Wu J J. Rotated Haar-Like features for face detection with in-plane rotation. In: Proceedings of the 12th International Conference, Virtual Systems and Multimedia (VSMM). Xi'an, China: Springer, 2006. 128-137
    [30] Du S Y, Liu J, Liu Y H, Zhang X T, Xue J R. Precise glasses detection algorithm for face with in-plane rotation. Multimedia Systems, 2017, 23(3):293-302 doi: 10.1007/s00530-015-0483-4
    [31] Wayne I, Langley P. Induction of one-level decision trees. In: Proceedings of the 9th International Workshop on Machine Learning. San Francisco, CA, USA: Morgan Kaufmann, 1992. 233-240
    [32] Viola P, Jones M J. Robust real-time face detection. International Journal of Computer Vision, 2004, 57(2):137-154 doi: 10.1023/B:VISI.0000013087.49260.fb
    [33] Chang C C, Lin C J. Training v-support vector classifiers:theory and algorithms. Neural Computation, 2001, 13(9):2119-2147 doi: 10.1162/089976601750399335
  • 加载中
图(4) / 表(6)
计量
  • 文章访问数:  2149
  • HTML全文浏览量:  216
  • PDF下载量:  805
  • 被引次数: 0
出版历程
  • 收稿日期:  2016-06-13
  • 录用日期:  2016-12-10
  • 刊出日期:  2018-04-20

目录

    /

    返回文章
    返回