2.845

2023影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

弱对齐的跨光谱人脸检测

闫梦凯 钱建军 杨健

闫梦凯, 钱建军, 杨健. 弱对齐的跨光谱人脸检测. 自动化学报, 2023, 49(1): 135−147 doi: 10.16383/j.aas.c210058
引用本文: 闫梦凯, 钱建军, 杨健. 弱对齐的跨光谱人脸检测. 自动化学报, 2023, 49(1): 135−147 doi: 10.16383/j.aas.c210058
Yan Meng-Kai, Qian Jian-Jun, Yang Jian. Weakly aligned cross-spectral face detection. Acta Automatica Sinica, 2023, 49(1): 135−147 doi: 10.16383/j.aas.c210058
Citation: Yan Meng-Kai, Qian Jian-Jun, Yang Jian. Weakly aligned cross-spectral face detection. Acta Automatica Sinica, 2023, 49(1): 135−147 doi: 10.16383/j.aas.c210058

弱对齐的跨光谱人脸检测

doi: 10.16383/j.aas.c210058
基金项目: 国家自然科学基金(61876083), 国家自然科学基金联合基金(U-1713208)资助
详细信息
    作者简介:

    闫梦凯:南京理工大学计算机科学与工程学院博士研究生. 主要研究方向为生物生理信息测量和计算机视觉. E-mail: ymk@njust.edu.cn

    钱建军:南京理工大学计算机科学与工程学院副教授. 2014年获南京理工大学博士学位. 主要研究方向为模式识别和计算机视觉. 本文通信作者. E-mail: csjqian@njust.edu.cn

    杨健:南京理工大学计算机科学与工程学院教授. 2002年获南京理工大学博士学位. 主要研究方向为模式识别, 计算机视觉和机器学习. E-mail: csjyang@njust.edu.cn

Weakly Aligned Cross-spectral Face Detection

Funds: Supported by National Natural Science Foundation of China (61876083) and Joint Fund of National Natural Science Foundation of China (U1713208)
More Information
    Author Bio:

    YAN Meng-Kai Ph.D. candidate at the School of Computer Science and Engineering, Nanjing University of Science and Technology. His research interest covers biophysiological information measurement and computer vision

    QIAN Jian-Jun Associate professor at the School of Computer Science and Engineering, Nanjing University of Science and Technology. He received his Ph.D. degree from Nanjing University of Science and Technology in 2014. His research interest covers pattern recognition and computer vision. Corresponding author of this paper

    YANG Jian Professor at the School of Computer Science and En-gineering, Nanjing University of Science and Technology. He received his Ph.D. degree from Nanjing University of Science and Technology in 2002. His research interest covers pattern recognition, computer vision and machine learning

  • 摘要: 跨光谱人脸检测在活体人脸识别、体温筛查等领域有着重要的应用价值. 众所周知, 可见光人脸易于检测, 然而红外人脸难于检测, 因此借助可见光图像的人脸检测结果进而完成红外人脸检测是一种有效的解决方案. 但是跨光谱图像之间不可避免的存在偏差, 导致检测精度不高. 为了解决这一问题, 提出了一种弱对齐跨光谱图像的人脸检测算法, 该方法基于跨光谱图像之间的偏差设计了候选框布置策略, 并在此基础上提出了跨光谱特征表示方法用于选取最优候选框. 此外, 本文还构建了一个跨光谱人脸数据集. 最后, 在跨光谱人脸数据集和OTCBVS人脸数据集上的实验结果证明, 该方法能够较好地完成红外图像人脸检测任务.
  • 图  1  跨光谱人脸检测

    Fig.  1  Cross-spectral face detection

    图  2  双相机与空间内任意一点的关系

    Fig.  2  The relationship between dual cameras and any point in space

    图  3  空间中任意一点在相机中的成像坐标

    Fig.  3  The imaging coordinates of any point in space in the camera

    图  4  像素坐标系与图像坐标系的关系

    Fig.  4  The relationship between pixel coordinate system and image coordinate system

    图  5  不同深度下的跨光谱人脸图像

    Fig.  5  Cross-spectral face images at different depths

    图  6  含有运动目标的跨光谱人脸图像

    Fig.  6  Cross-spectral face images with moving target

    图  7  人脸高度与其成像高度的关系

    Fig.  7  Relationship between face height and image height

    图  8  跨光谱人脸检测框架

    Fig.  8  Cross-spectral face detection framework

    图  9  跨光谱特征表示网络

    Fig.  9  Cross-spectral feature representation network

    图  10  跨光谱特征表示网络训练方式

    Fig.  10  Cross-spectral feature representation network training method

    图  11  含有部分人脸的负样本

    Fig.  11  Improved negative sample selection method

    图  12  相机安装位置

    Fig.  12  Camera installation location

    图  13  不同采集条件下的图像

    Fig.  13  Images under different acquisition conditions

    图  14  检测结果对比图

    Fig.  14  Comparison of face detection results

    表  1  测试集为CSF-白天的实验结果

    Table  1  Experiment results on CSF-day

    算法IoU > 0.5 时 AP (%)IoU > 0.3 时 AP (%)
    坐标映射44.688.4
    粗略纠正55.987.9
    本文算法87.589.6
    下载: 导出CSV

    表  2  测试集为CSF-夜间的实验结果

    Table  2  Experiment results on CSF-night

    算法IoU > 0.5 时 AP (%)IoU > 0.3 时 AP (%)
    坐标映射36.982.7
    粗略纠正50.882.4
    本文算法81.884.1
    下载: 导出CSV

    表  3  测试集为OTCBVS的实验结果

    Table  3  Experiment results on OTCBVS

    算法IoU > 0.5 时 AP (%)IoU > 0.3 时 AP (%)
    坐标映射16.446.8
    粗略纠正54.576.5
    本文算法74.486.6
    下载: 导出CSV

    表  4  候选框召回率(%)

    Table  4  Proposal recall (%)

    数据集IoU > 0.5IoU > 0.3
    CSF96.998.4
    OTCBVS89.391.6
    下载: 导出CSV

    表  5  CSF中候选框的选取对模型的影响

    Table  5  Influence of the selection of the proposal on the model in CSF

    候选框IoU > 0.5 时 AP (%)时间 (ms)
    1/871.49
    1/8, 2/884.816
    1/8, 2/8, 3/886.323
    1/8, $\cdots , 4/8$86.428
    下载: 导出CSV

    表  6  OTCBVS中候选框的选取对模型的影响

    Table  6  Influence of the selection of the proposal on the model in OTCBVS

    候选框IoU > 0.5 时 AP (%)时间 (ms)
    1/870.510
    1/8, 2/873.116
    1/8, 2/8, 3/874.224
    1/8, $\cdots, 4/8$74.430
    下载: 导出CSV

    表  7  负样本类型对模型精度的影响

    Table  7  Effect of negative sample type on model accuracy

    负样本类型IoU > 0.5 时 AP (%)
    046.1
    4/870.5
    5/869.7
    6/847.1
    7/821.5
    0, 4/8, 5/8, 6/8, 7/886.4
    下载: 导出CSV

    表  8  CSF数据集上的对比实验结果

    Table  8  Comparative experiment results on CSF dataset

    算法IoU > 0.5 时 AP (%) IoU > 0.3 时 AP (%)
    FaceBoxes9.19.1
    S3FD9.19.1
    Pyramidbox35.936.3
    DSFD35.836.3
    Tinyface56.582.4
    S3FD-IR72.373.4
    DSFD-IR81.983.7
    DSFD-本文算法86.488.5
    下载: 导出CSV

    表  9  OTCBVS数据集上的对比实验结果

    Table  9  Comparative experiment results on OTCBVS dataset

    算法IoU > 0.5 时 AP (%) IoU > 0.3 时 AP (%)
    FaceBoxes
    S3FD9.19.1
    Pyramidbox36.136.1
    DSFD27.227.2
    Tinyface25.138.6
    S3FD-IR60.873.6
    DSFD-IR69.470.4
    DSFD-本文算法75.086.3
    下载: 导出CSV
  • [1] Bilodeau G A, Torabi A, St-Charles P L, Riahi D. Thermal–visible registration of human silhouettes: A similarity measure performance evaluation. Infrared Physics & Technology, 2014, 64: 79-86.
    [2] Rowley H A, Baluja S, Kanade T. Neural network-based face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(1), 23-38. doi: 10.1109/34.655647
    [3] Rowley H A, Baluja S, Kanade T. Rotation invariant neural network-based face detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Santa Barbara, USA: IEEE, 1998. 38−44
    [4] Viola P, Jones M J. Robust Real-Time Face Detection. International Journal of Computer Vision, 2004, 57(2): 137-154. doi: 10.1023/B:VISI.0000013087.49260.fb
    [5] Li S Z, Zhu L, Zhang Z, Blake A, Zhang H, Shum H. Statistical learning of multi-view face detection. In: Proceedings of the 7th European Conference on Computer Vision. Berlin, Germany: 2002. 67−81
    [6] Mathias M, Benenson R, Pedersoli M, Van Gool L. Face detection without bells and whistles. In: Proceedings of the 13rd European Conference on Computer Vision. Zurich, Switzerland: 2014. 720−735
    [7] Li Q, Sun Z, He R, Tan T. Learning symmetry features for face detection based on sparse group lasso. In: Proceedings of the Chinese Conference on Biometric Recognition. Jinan, China: 2013. 162−169
    [8] Li J, Wang Y, Wang C, Tai Y, Qian J, Yang J, et al. DSFD: Dual shot face detector. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020. 5055−5064
    [9] Zhang S, Zhu X, Lei Z, Shi H, Wang X, Li S Z, et al. S3FD: Single shot scale-invariant face detector. In: Proceedings of the IEEE International Conference on Computer Vision. Venice, Ita-ly: IEEE, 2017. 192−201
    [10] Bai Y, Zhang Y, Ding M, Ghanem B. Finding tiny faces in the wild with generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 21–30
    [11] Zhang S, Wen L, Shi H, Lei Z, Lyu S, Li S Z. Single-Shot Scale-Aware Network for Real-Time Face Detection. International Journal of Computer Vision, 2019, 127(6-7): 537-559. doi: 10.1007/s11263-019-01159-3
    [12] Zhang S, Zhu X, Lei Z, Shi H, Wang X, Li S Z. Faceboxes: A CPU real-time face detector with high accuracy. In: Proceedin-gs of the IEEE International Joint Conference on Biometrics. Denver, USA: IEEE, 2017. 1−9
    [13] Tang X, Du D K, He Z, Liu J. Pyramidbox: A context-assisted single shot face detector. In: Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: 2018. 797− 813
    [14] Yang S, Luo P, Loy C C, Tang X. Wider face: A face detection benchmark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 525−5533
    [15] C. Ma, T Ngo, U Hideaki, N Hajime, S. Atsushi, T Rin-Ichiro. Adapting local features for face detection in thermal image. Sensors, 2017, 17(12): 2741-.
    [16] Kyal C K, Poddar H, Reza M. Detection of human face by thermal infrared camera using MPI model and feature extraction method. In: Proceedings of the 4th International Conference on Computing Communication and Automation. Greater Noida, India: IEEE, 2018. 1−5
    [17] Budzan S, Wyżgolik R. Face and eyes localization algorithm in thermal images for temperature measurement of the inner canthus of the eyes. Infrared Physics & Technology, 2013, 60: 225-234.
    [18] Ribeiro R F, Fernandes J M, Neves A J R. Face detection on infrared thermal image. In: Proceedings of the 2nd International Conference on Advances in Signal Image and Video Processing. Barcelona, Spain: 2017. 38–42
    [19] Goulart C, Valadão C, Delisle-Rodriguez D, Funayama D, Favarato A, Baldo G, et al. Visual and Thermal Image Processing for Facial Specific Landmark Detection to Infer Emotions in a Child-Robot Interaction. Sensors, 2019, 19(13): 2844. doi: 10.3390/s19132844
    [20] Somboonkaew A, Prempree P, Vuttivong S, Wetcharungsri J, Porntheeraphat S, Chanhorm S, et al. Mobile-platform for automatic fever screening system based on infrared forehead temperature. In: Proceedings of the IEEE Opto-Electronics and Communications Conference and Photonics Global Conference. Sing-apore, Singapore: IEEE, 2017. 1−4
    [21] Mallat K, Dugelay J L. A benchmark database of visible and thermal paired face images across multiple variations. In: Proceedings of the IEEE International Conference of the Biometrics Special Interest Group. Darmstadt, Germany: IEEE, 2018. 1−5
    [22] Panetta K, Wan Q, Agaian S, Rajeev S, Kamath S, Rajendran R, et al. A comprehensive database for benchmarking imaging systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 42(3): 509-520.
    [23] Hwang S, Park J, Kim N, Choi Y, So Kweon I. Multispectral pedestrian detection: Benchmark dataset and baseline. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015. 1037−1045
    [24] 张秀伟, 张艳宁, 杨涛, 等. 基于Co-motion的可见光--热红外图像序列自动配准算. 自动化学报, 2010, 36(009): 1220-1231. doi: 10.3724/SP.J.1004.2010.01220

    Zhang Xiu-Wei, Zhang Yan-Ning, Yang Tao, Zhang Xin-Gong, Shao Da-Pei. Automatic Visual-thermal Image Sequence Registration Based on Co-motion. Acta Automatica Sinica, 2010, 36(009): 1220-1231. doi: 10.3724/SP.J.1004.2010.01220
    [25] 高雪琴, 刘刚, 肖刚, 等. 基于FPDE的红外与可见光图像融合算法[J]. 自动化学报, 2020, v. 46(04): 186-194.

    Gao Xue-Qin, Liu Gang, Xiao Gang, Bavirisetti Durga Prasad, SHI Kai-Lei. Fusion Algorithm of Infrared and Visible Images Based on FPDE. Acta Automatica Sinica, 2020, v. 46(04): 186-194.
    [26] 廉蔺, 李国辉, 张军, 涂丹. 基于边缘最优映射的红外和可见光图像自动配准算. 自动化学报, 2012, 38(04): 570-581. doi: 10.3724/SP.J.1004.2012.00570

    Lian Lin, Li Guo-Hui, Zhang Jun, Tu Dan. An Automatic Registration Algorithm of Infrared and Visible Images Based on Optimal Mapping of Edges. Acta Automatica Sinica, 2012, 38(04): 570-581. doi: 10.3724/SP.J.1004.2012.00570
    [27] (刘松涛, 刘振兴, 姜宁. 基于融合显著图和高效子窗口搜索的红外目标分. 自动化学报, 2018, 44(012): 2210-2221).

    Liu Song-Tao, Liu Zhen-Xing, Jiang Ning. Target Segmentation of Infrared Image Using Fused Saliency Map and Efficient Subwindow Search. Acta Automatica Sinica, 2018, 44(012): 2210-2221.
    [28] (袁浩期, 李扬, 王俊影, 等. 基于红外热像的行人面部温度高精度检测技术. 红外技术, 2019, v. 41;No. 324(12): 94-99).

    Yuan Hao-qi, Li Yang, Wang Jun-Ying, Liu Hang. High Precision Detection Technology of Pedestrian Face Temperature Based on Infrared Thermal Imaging. Infrared Technology, 2019, v. 41;No. 324(12): 94-99.
    [29] Zhi T, Pires B R, Hebert M, Narasimhan S. G. Deep material-aware cross-spectral stereo matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 1916−1925
    [30] Liang M, Guo X, Li H, Wang X, Song, Y. Unsupervised cross-spectral stereo matching by learning to synthesize. In: Proceedings of the AAAI Conference on Artificial Intelligence. Honolulu, USA: 2019. 8706−8713
    [31] Zhang L, Zhu X, Chen X, Yang X, Lei Z, Liu Z. Weakly aligned cross-modal learning for multispectral pedestrian detection. In: Proceedings of the IEEE International Conference on Computer Vision. Seoul, South Korea: IEEE, 2019. 5127−5137
    [32] Dwith Chenna Y N, Ghassemi P, Pfefer T J, Casamento J, Wang Q. Free-Form Deformation Approach for Registration of Visible and Infrared Facial Images in Fever Screening. Sensors, 2018, 18(2): 125. doi: 10.3390/s18010125
    [33] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint, 2014, arXiv: 1409.1556
    [34] Schroff F, Kalenichenko D, Philbin J. FaceNet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015. 815−823
  • 加载中
图(14) / 表(9)
计量
  • 文章访问数:  1394
  • HTML全文浏览量:  350
  • PDF下载量:  220
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-01-20
  • 网络出版日期:  2021-07-19
  • 刊出日期:  2023-01-07

目录

    /

    返回文章
    返回