Image Splicing Detection Based on Spatial Lighting Consistency Analysis Under Perspective Projection
-
摘要: 将一个人的头像剪切并拼接到另一张照片中，是一种常见的图像篡改手段.如果将该合成照片用于敲诈勒索，会对社会带来严重危害.因此，用来检测图像篡改的图像取证技术具有重大意义.由于不同照片成像环境不同，拼接时很难做到不同人脸的光照绝对一致，因此可以通过光照是否一致检测篡改.以往光照估计方法基于平行投影的假设，利用照片投影光照进行光照一致性分析.实际上，相机针孔模型是透视投影，从而导致上述检测方法出现误差.针对这一问题，本文提出一种透视投影下物体空间光照估计算法，将各人脸姿态统一到相机坐标系下，估计各人脸相对于相机坐标系的空间光照，然后分析空间光照一致性.另外，根据人脸空间光照一致性约束可以优化出相机参数，并得到该参数下的等效焦距、人脸空间位置及重新透视投影的图像等空间信息.本文将空间光照的一致性和上述空间信息的合理性作为依据，对人脸图像进行拼接篡改检测.实验结果表明，相比于传统方法基于平行投影光照进行光照一致性分析，采用本文提出的方法得到的空间光照进行光照一致性分析具有更高的准确度，结合相关信息进行照片空间合理性分析的篡改检测方法具有更强的说服力.Abstract: Splicing a face image of a person into another photo is a common tempering way, and it is harmful once used for blackmail. Therefore, the image forensics for tampered image detection is of great significance. As different photos are taken in different environments with different cameras, it is very difficult to ensure the spliced faces have absolutely consistent light. Thus the illumination consistency can be an effective clue for tempering detection. Existing illumination consistency analysis based tempering detection methods perform the illumination estimation under the assumption of parallel projection. However, the results usually have huge but subtle errors because the camera pinhole model is perspective projection. To solve this problem, we propose a spatial lighting estimation method under perspective projection for the illumination consistency analysis. In addition, the spatial information analysis is combined to obtain more reasonable detection results. Specifically, the coordinate system of each face is firstly unified into the camera coordinate system and the spatial lighting of each face relative to the camera coordinate system is estimated. Then the camera parameters are optimized according to the consistency constraint of faces' spatial lighting and the spatial information including the equivalent focal length, face spatial position and re-perspective projection image under the camera parameters are calculated. Finally, the consistency of spatial lighting and the rationality of the spatial information are both taken as clues to detect face image splicing forgery. The experimental results show that compared with previous methods, the spatial lighting estimation method has higher accuracy, and the spatial information analysis method has stronger persuasion.
-
Key words:
- Image forensics /
- lighting estimation /
- spatial lighting /
- perspective projection /
- splicing detection
1) 本文责任编委 黄庆明 -
图 1 被质疑造假的日本人质视频截图
Fig. 1 The photo of Japanese hostages which is considered as a spliced image
图 3 平行投影和透视投影对光照估计的影响示意图
Fig. 3 The influences of the projection methods on the lighting estimation
图 4 人脸透视形变程度随人脸到相机距离变化的情况
Fig. 4 The influences of the distance from the face to the camera on the facial perspective distortion
图 6 固定人脸和相机光心, 人脸的照片姿态就保持不变, 不随相机焦距变化和姿态旋转而变化
Fig. 6 The poses of the face in the photo remain the same once the face and the optical center are fixed
图 9 晴天下拍摄的一个足球队的真实照片
Fig. 9 A pristine photo of a football team taken on a sunny day to verify our proposed approach
图 11 图 9中18名队员的三维人脸模型和投影光照渲染球, 以及三组通过不同人脸估计的透视参数对应的人脸空间姿态、空间光照渲染球及重新投影的人脸透视模型对比
Fig. 11 The 3D face models and projected lighting of the 18 players in Fig. 9, and three groups of spatial pose, spatial lighting and re-projected face model under the estimated perspective parameters using three different pairs of faces
图 12 根据人脸(6, 11)、(3, 12)、(5, 10)优化得到的参数估计出在人脸空间分布(上方为正视图, 下方为俯视图)
Fig. 12 The estimated spatial poses of 18 human faces according to the optimized parameters of human faces (6, 11), (3, 12) and (5, 10), respectively (The first row is the face view, and the second row is the top view of the faces)
图 13 对DSO-1数据集中的四幅样本图像的检测结果. (a)$\sim$(d)分别是对拼接图像正确检测, 对原始图像错误报警, 对原始图像正确检测, 对拼接图像错误检测
Fig. 13 The detection results of our method on four sample images in the DSO-1 dataset. (a) $\sim$ (d) are respectively a correct detection for splicing image, a false alarm for pristine image, a correct detection for pristine image and a miss detection for splicing image
表 1 实验中各案例相关参数列表及判断意见
Table 1 Comparisons of relevant parameters and corresponding judgment opinions of each case in the experiment
人脸组合 $d(\boldsymbol{l}_{pij})$ 意见1 $d(\boldsymbol{l}_{sij})$ 意见2 $F_{eq}$ (mm)/理论焦距 $D_{HH}\, {\rm (m)}/D_{OH}$ (m) $I_{re}$ 意见3 实际 图 10(6, 11) 0.0219 不一致 0.0001 一致 29.3/中 0.50合理/4.00合理 接近 合理 真实 图 10(3, 12) 0.3508 不一致 0.3009 不一致 5/中 0.18过近/1.00过近 严重 不合理 篡改 图 10(5, 10) 0.0063 一致 0.0098 一致 115/中 1.00过远/15.00过远 接近 不合理 真实 图 14(a) 0.2463 不一致 0.2371 不一致 125/中 2.00过远/20.00过远 接近 不合理 篡改 图 14(b) 0.0093 一致 0.0065 一致 94/短 0.60过远/18.00过远 接近 不合理 真实 图 14(c) 0.0635 不一致 0.0167 一致 20.2/中 0.24合理/1.00合理 接近 合理 真实 图 14(d) 0.1065 不一致 0.0361 不一致 34/中 0.69合理/1.70合理 接近 合理 篡改 图 1 0.1327 不一致 0.0755 不一致 44/中 1.20过远/3.00合理 接近 合理 待检 
下载: 导出CSV
-
[1] Rossler A, Cozzolino D, Verdoliva L, Riess C, Thies J, Niessner M. FaceForensics++: learning to detect manipulated facial images. arXiv preprint, 2019. arXiv: 1901.08971 [2] Farid H. Creating and Detecting Doctored and Virtual Images: Implications to the Child Pornography Prevention Act. Technical Report TR2004-518, Department of Computer Science, Dartmouth College, USA, 2004. [3] Farid H. A survey of image forgery detection. IEEE Signal Processing Magazine, 2009, 26(2): 16-25 [4] Popescu A C, Farid H. Exposing digital forgeries by detecting traces of resampling. IEEE Transactions on Signal Processing, 2005, 53(2): 758-767 doi: 10.1109/TSP.2004.839932 [5] Luo W, Huang J, Qiu G. JPEG error analysis and its applications to digital image forensics. IEEE Transactions on Information Forensics and Security, 2010, 5(3): 480-491 doi: 10.1109/TIFS.2010.2051426 [6] Gou H, Swaminathan A, Wu M. Noise features for image tampering detection and steganalysis. In: Proceedings of the 2007 IEEE International Conference on Image Processing. San Antonio, TX, USA: IEEE, 2007. 97-100 https://ieeexplore.ieee.org/document/4379530 [7] Zhou P, Han X, Morariu V I, Davis L S. Learning rich features for image manipulation detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 1053-1061 https://www.researchgate.net/publication/325143175_Learning_Rich_Features_for_Image_Manipulation_Detection [8] Gloe T, Franz E, Winkler A. Forensics for flatbed scanners. In: Proceedings of SPIE 6505, Security, Steganography, and Watermarking of Multimedia Contents IX. California, USA: International Society for Optics and Photonics, 2007. 650511 [9] Kakar P, Sudha N, Ser W. Exposing digital image forgeries by detecting discrepancies in motion blur. IEEE Transactions on Multimedia, 2011, 13(3): 443-452 doi: 10.1109/TMM.2011.2121056 [10] Farid H, Bravo M J. Image forensic analyses that elude the human visual system. In: Proceedings of SPIE 7541, Media Forensics and Security Ⅱ. California, USA: International Society for Optics and Photonics, 2010. 754106 doi: 10.1117/12.837788 [11] Peng B, Wang W, Dong J, Tan T. Image forensics based on planar contact constraints of 3D objects. IEEE Transactions on Information Forensics and Security, 2018, 13(2): 377-392 doi: 10.1109/TIFS.2017.2752728 [12] Carvalho T J D, Riess C, Angelopoulou E, Pedrini H, Rocha A D R. Exposing digital image forgeries by illumination color classification. IEEE Transactions on Information Forensics and Security, 2014, 18(6): 1182-1194 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=Doaj000003806817 [13] Johnson M K, Farid H. Exposing digital forgeries through specular highlights on the eye. In: Proceedings of International Workshop on Information Hiding. Berlin, Heidelberg, Germany: Springer, 2007. 311-325 doi: 10.1007/978-3-540-77370-2_21 [14] Liu Q, Cao X, Chao D, Guo X. Identifying image composites through shadow matte consistency. IEEE Transactions on Information Forensics and Security, 2011, 6(3): 1111-1122 doi: 10.1109/TIFS.2011.2139209 [15] Kee E, O0Brien J F, Farid H. Exposing photo manipulation with inconsistent shadows. ACM Transactions on Graphics, 2013, 32(3): 28 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=33fd7688600a6dd746ed39aa3727a855 [16] Kee E, O0Brien J F, Farid H. Exposing photo manipulation from shading and shadows. ACM Transactions on Graphics, 2014, 33(5): 1-21 https://www.mendeley.com/catalogue/exposing-photo-manipulation-shading-shadows/ [17] Johnson M K, Farid H. Exposing digital forgeries by detecting inconsistencies in lighting. In: Proceedings of the 7th workshop on Multimedia and security. NewYork, NY, USA: ACM Press, 2005. 1-10 https://www.mendeley.com/catalogue/exposing-digital-forgeries-detecting-inconsistencies-lighting/ [18] Carvalho T, Farid H, Kee E R. Exposing photo manipulation from user-guided 3d lighting analysis. In: Proceedings of SPIE Symposium on Electronic Imaging. California, USA: International Society for Optics and Photonics, 2015. 940902 [19] Johnson M K, Farid H. Exposing digital forgeries in complex lighting environments. IEEE Transactions on Information Forensics and Security, 2007, 2(3): 450-461 doi: 10.1109/TIFS.2007.903848 [20] Kee E, Farid H. Exposing digital forgeries from 3-D lighting environments. In: Proceedings of IEEE International Workshop on Information Forensics and Security. Seattle, USA: IEEE, 2010. 1-6 [21] Peng B, Wang W, Dong J, Tan T. Optimized 3D lighting environment estimation for image forgery detection. IEEE Transactions on Information Forensics and Security, 2017, 12(2): 479-494 doi: 10.1109/TIFS.2016.2623589 [22] 孙鹏, 郎宇博, 樊舒, 沈喆, 彭思龙, 刘磊.图像拼接篡改的自动色温距离分类检验方法.自动化学报, 2018, 44(7): 171-182 http://www.aas.net.cn/CN/abstract/abstract19318.shtmlSun Peng, Lang Yu-Bo, Fan Shu, Shen Zhe, Peng Si-Long, Liu Lei. Detection of image splicing manipulation by automated classification of color temperature distance. Acta Automatica Sinica, 2018, 44(7): 171-182 http://www.aas.net.cn/CN/abstract/abstract19318.shtml [23] 朱叶, 申铉京, 陈海鹏.基于彩色LBP的隐蔽性复制–粘贴篡改盲鉴别算法.自动化学报, 2017, 43(3): 390-397 http://www.aas.net.cn/CN/abstract/abstract19017.shtmlZhu Ye, Shen Xuan-Jing, Chen Hai-Peng. Covert copy-move forgery detection based on color LBP. Acta Automatica Sinica, 2017, 43(3): 390-397 http://www.aas.net.cn/CN/abstract/abstract19017.shtml [24] Blanz V, Vetter T. A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th annual conference on Computer graphics and interactive techniques. NewYork, USA: ACM, 1999. 187-194 [25] Zhu X, Lei Z, Yan J, Yi D, Li S Z. High-fidelity pose and expression normalization for face recognition in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015. 787-796 [26] King D E. Dlib-ml: a machine learning toolkit. Journal of Machine Learning Research, 2009, 10(Jul): 1755-1758 https://core.ac.uk/display/21155954 [27] Phong B T. Illumination for computer generated pictures. Communications of the Acm, 1975, 18(6): 311-317 doi: 10.1145/360825.360839 [28] Ramamoorthi R, Hanrahan P. On the relationship between radiance and iIrradiance: determining the illumination from images of a convex lambertian object. Journal of the Optical Society of America A, 2001, 18(10): 2448-2459 doi: 10.1364/JOSAA.18.002448 [29] Basri R, Jacobs D. Lambertian reflectance and linear subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(2): 218-233 doi: 10.1109/TPAMI.2003.1177153 [30] Zhang Z. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334 doi: 10.1109/34.888718 [31] Zhou W, Kambhamettu C. Estimation of illuminant direction and intensity of multiple light sources. In: Proceedings of European conference on computer vision. Berlin, Heidelberg, Germany: Springer, 2002. 206-220 -
点击查看大图
计量
- 文章访问数: 1417
- HTML全文浏览量: 420
- PDF下载量: 121
- 被引次数: 0
