-
摘要: 仿射类抠像方法主要分为KNN (K-nearest neighbor)类和Matting Laplacian类方法, 本文结合这2种方法的优点提出了一种基于仿射类的双层次抠像方法.其中, 第一层为绝对像素的划分层次或预处理层次, 采用了基于KNN类简单权重与相对远距离的搜索方法, 并结合初始Trimap未知区域大小无关的方式; 第二层为混合像素的计算层次或最终抠像层次, 充分利用了第一层计算获得的剩余混合像素的宽度, 自适应地调整Matting Laplacian中的颜色线性模型所构成颜色近邻的核宽度.每个层次均按图像的全局颜色重叠程度相应调整合理的搜索范围.本文的实验具备以下特点: 1)预处理层次之后采用了若干典型的后续抠像方法, 以展现本文方法相比于其他预处理方法对后续抠像操作步骤的优越性和兼容性; 2)最终抠像层次引入了若干其他抠像方法, 以验证本文抠像方法的优越性.实验表明, 相比于其他单层次的仿射类方法, 无论对于计算绝对像素还是混合像素, 本文方法都可以大幅提升计算结果的准确率.
-
关键词:
- 图像抠像 /
- 仿射类抠像 /
- Matting Laplacian /
- KNN搜索 /
- 颜色线性模型
Abstract: Affinity based image matting methods can be categorized into KNN (K-nearest neighbor) based matting and matting Laplacian based matting, and this paper raises a hierarchical framework on affinity based matting according to the analyses of the advantages of these two popular affinity based image matting methods. The first opaque pixel classification layer, also named as pre-processing layer, employs a relatively far searching fashion based on simple weights in KNN and is spatial irrelevant to the unknown region of the initial Trimap. The second mixed pixel computation layer, also named as final matting layer, adaptively adjusts the kernel size of the color line model in matting Laplacian according to the remaining size of the unknown region. Each layer adjusts proper searching range adaptively according to the overlapping degree between global foreground and background colors. The following distinctions are provided in the experiments. First, several representative matting algorithms are processed after the first layer to show the superiority and compatibility of our pre-processing method over other pre-processing methods. Second, several alternative matting methods are also processed after the first layer to show the superiority of our final matting method over other matting methods. Experimental results show that our approach can greatly raise the solving precisions for both opaque and mixed pixels.-
Key words:
- Image matting /
- affinity based matting /
- matting Laplacian /
- KNN (K-nearest neighbor) searching /
- color line model
1) 本文责任编委 刘跃虎 -
图 1 图像抠像问题的基本输入输出和本文的双层次抠像结构
Fig. 1 Input and output of image matting and the hierarchical framework of our method
图 2 仿射类方法的2种分类方式对应的算法及适用范围(其中Lap表明在原简单权重方法的基础上采用Matting Laplacian)
Fig. 2 Two types of affinity based matting method and the corresponding algorithms and application scopes (where Lap denotes the application of matting Laplacian based on simple weight methods)
图 3 各$\alpha^{\rm{true}}$区间中5种传统仿射类算法$\alpha$结果的平均MSE比较(其中$x$轴坐标中的0.0表示$0.0\sim0.05$区间等)
Fig. 3 MSE comparison on five traditional affinity based matting algorithms in each $\alpha^{\rm{true}}$ interval (where 0.0 in $x$-label denotes the range of $0.0\sim0.05$, etc)
图 4 以背景为例, 未知点的各种搜索方式与搜索范围
Fig. 4 Various kinds of searching manners and searching ranges for an unknown pixel
图 5 全局低重合度图像中本文预处理方法与其他2种单一搜索方式的结果比较(前2列与后2列分别显示了远距离与近距离搜索方法较好的例子, 图像中的线条表示前景与背景边界)
Fig. 5 Comparison of pre-processing results between our method and two unary searching methods in low global overlapped cases (where the first and last two rows show good results in methods with long and short searching ranges respectively, and the lines in the input images show known foreground and background boundaries)
图 6 全局高重合度图像中本文预处理方法与其他2种单一搜索方式的结果比较
Fig. 6 Comparison of pre-processing results between our method and two unary searching methods in high global overlapped cases
图 8 9种预处理方法与无预处理方法在一些局部图像中的代价比较($\times10^{-4}$)
Fig. 8 Comparison on cost of several local images in 9 pre-processing methods and no pre-processing method ($\times10^{-4}$)
图 9 3种型号的Trimap下, 对所有训练图像抠像结果的平均MSE比较, 格式为:算法名称$^{\text{排名}}$, 排名为在8种后续抠像算法下, 每种预处理方法在10种预处理方法中的平均排名
Fig. 9 Average MSE Comparisons on the matting results of all the training images over three types of Trimap, where the format is [algorithm name]$^{\rm{rank}}$ and the rank denotes the average rank for each of the pre-processing methods out of 10 over the 8 matting methods
图 10 对于图 7的前6个局部图像及预处理后的Trimap, 采用10种方法进行抠像计算后的MSE比较, 其中前2个例子为硬边界, 中间2个例子为软边界, 后2个例子为长毛发边缘与前景空洞, 且这些例子中未知区域的宽度也逐渐增大
Fig. 10 MSE comparison on matting results of the first 6 local images in Fig. 7 for 10 matting methods, where the first, median, and last 2 cases are hard boundaries, soft boundaries, and long hair edges and foreground holes respectively, in which the sizes of unknown regions gradually enlarge
图 11 3种型号的Trimap下, 在各$\alpha$区间上, 10种混合像素计算方法对所有训练图像的抠像结果的平均MSE比较, 其中$x$坐标轴中的0.15表示$0.15\sim0.25$区间等
Fig. 11 Average MSE comparisons on matting results of all the training images for 10 matting methods over 3 types of Trimap in each $\alpha$ range, where 0.15 in $x$-label denotes the range of $0.15\sim0.25$, etc.
图 12 10种仿射类抠像方法的3个实例
Fig. 12 3 Cases of matting results for 10 a–nity based matting methods
表 1 9种预处理方法与未预处理方法在3种型号Trimap之下对所有训练图像的代价值之和的比较
Table 1 Comparison on sum cost of all the training images within 9 pre-processing methods and no pre-processing method over three types of Trimap
预处理方法 代价值($\times10^{-3}$)Trimap型号 巨大 大 小 无预处理 36.6 22.4 13.7 Large Kernel 14.6 11.0 8.3 Nonlocal 17.3 10.2 6.8 Closed Form 16.4 10.4 6.6 KNN-0.5 16.3 9.4 5.5 KNN 14.6 9.1 5.8 KNN-0.01 11.4 8.5 5.9 CCM 10.3 8.4 6.3 KNN-0.1 9.8 6.3 4.7 本文方法 9.3 6.3 4.6 
下载: 导出CSV
表 2 10种仿射类方法最终抠像结果的MSE比较
Table 2 Final MSE comparison on matting results for 10 affinity based matting methods
算法 MSE($\times10^{-2}$)&Trimap型号 巨大 大 小 Nonlocal 5 13.9 8.8 5.2 Nonlocal 10 10.5 6.7 4.0 CCM 7.6 6.1 3.8 KNN-0.01 7.5 5.6 2.4 Nonlocal 20 7.5 4.7 3.0 Closed Form 6.8 4.5 2.6 KNN-0.5 6.1 3.8 2.4 KNN 5.8 4.0 2.3 KNN-0.1 4.9 3.6 2.4 本文方法 3.2 2.5 2.2
下载: 导出CSV
-
[1] Levin A, Lischinski D, Weiss Y. A closed-form solution to natural image matting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2): 228-242 doi: 10.1109/TPAMI.2007.1177 [2] He K M, Sun J, Tang X O. Fast matting using large kernel matting Laplacian matrices. In: Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE, 2010. 2165-2172 [3] Lee P, Wu Y. Nonlocal matting. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA: IEEE, 2011. 2193-2200 [4] Chen Q F, Li D Z Y, Tang C K. KNN matting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(9): 2175-2188 doi: 10.1109/TPAMI.2013.18 [5] Shi Y F, Au O C, Pang J H, Tang K T, Sun W X, Zhang H, et al. Color clustering matting. In: Proceedings of the 2013 IEEE International Conference on Multimedia and Expo. San Jose, USA: IEEE, 2013. 1-6 [6] Grady L, Schiwietz T, Aharon S, Westermann R. Random walks for interactive alpha matting. In: Proceedings of the 2005 International Conference on Visualization, Imaging and Image Processing. Benidorm, Spain: ACTA, 2005. 423-429 [7] Zhang Z P, Zhu Q S, Xie Y Q. Learning based alpha matting using support vector regression. In: Proceedings of the 2012 19th IEEE International Conference on Image Processing, Orlando, USA: IEEE, 2012. 2109-2112 [8] Chuang Y Y, Curless B, Salesin D H, Szeliski R. A Bayesian approach to digital matting. In: Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recognition. Kauai, USA: IEEE, 2001. 264-271 [9] Wang J, Cohen M F. An iterative optimization approach for unified image segmentation and matting. In: Proceedings of the 10th IEEE International Conference on Computer Vision. Beijing, China: IEEE, 2005. 936-943 [10] Wang J, Cohen M F. Optimized color sampling for robust matting. In: Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, USA: IEEE, 2007. 1-8 [11] Rhemann C, Rother C, Gelautz M. Improving color modeling for alpha matting. In: Proceedings of British Machine Vision Conference. Leeds, UK: BMVA, 2008. 1155-1164 [12] He K M, Rhemann C, Rother C, Tang X O, Sun J. A global sampling method for alpha matting. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA: IEEE, 2011. 2049-2056 [13] Shahrian E, Rajan D, Price B, Cohen S. Improving image matting using comprehensive sampling sets. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, USA: IEEE, 2013. 636-643 [14] Varnousfaderani E S, Rajan D. Weighted color and texture sample selection for image matting. IEEE Transactions on Image Processing, 2013, 22(11): 4260-4270 doi: 10.1109/TIP.2013.2271549 [15] Karacan L, Erdem A, Erdem E. Image Matting with KL-Divergence Based Sparse Sampling. In: Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Chile: IEEE, 2015. 424-432 [16] Gastal E S L, Oliveira M M. Shared sampling for real-time alpha matting. Computer Graphics Forum, 2010, 29(2): 575-584 doi: 10.1111/j.1467-8659.2009.01627.x [17] Rhemann C, Rother C, Rav-Acha A, Sharp T. High resolution matting via interactive trimap segmentation. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, USA: IEEE, 2008. 1-8 [18] 姚桂林, 姚鸿勋.基于仿射方法的图像抠图算法综述. 计算机辅助设计与图形学学报, 2016, 28(4): 677-692 doi: 10.3969/j.issn.1003-9775.2016.04.019Yao Gui-Lin, Yao Hong-Xun. A Review on Affinity Based Algorithms in Image Matting. Journal of Computer-Aided Design & Computer Graphics, 2016, 28(4): 677-692 doi: 10.3969/j.issn.1003-9775.2016.04.019 [19] Johnson J, Varnousfaderani E S, Cholakkal H, Rajan D. Sparse Coding for Alpha Matting. IEEE Transactions on Image Processing, 2016, 25(7): 3032-3043 doi: 10.1109/TIP.2016.2555705 [20] Cho D, Tai Y W, Kweon I S. Deep convolutional neural network for natural image matting using initial alpha mattes. IEEE Transactions on Image Processing, 2019, 28(3): 1054-1067 doi: 10.1109/TIP.2018.2872925 [21] Shen X Y, Tao X, Gao H Y, Zhou C, Jia J Y. Deep automatic portrait matting. In: Proceedings of the 14th European Conference on Computer Vision. Amsterdam, Netherlands: Springer, 2016. 92-107 [22] Xu N, Price B, Cohen S, Huang T. Deep image matting. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 311-320 [23] Rhemann C, Rother C, Wang J, Gelautz M, Kohli P, Rott P. A perceptually motivated online benchmark for image matting. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA: IEEE, 2009. 1826-1833 [24] Yao G L. A survey on pre-processing in image matting. Journal of Computer Science and Technology, 2017, 32(1): 122-138 doi: 10.1007/s11390-017-1709-z [25] Yao G L, Zhao Z J, Liu S H. A comprehensive survey on sampling based image matting. Computer Graphics Forum. 2017, 36(8): 613-628 doi: 10.1111/cgf.13156 [26] Chen X, He F Z, Yu H P. A matting method based on full feature coverage. Multimedia Tools and Applications, 2018: 1-29 [27] 姚桂林, 刘绍辉, 李敏, 姚鸿勋, 胡文.后处理在数字抠像中的应用与解析. 电子学报, 2017, 45(3): 719-729 doi: 10.3969/j.issn.0372-2112.2017.03.032Yao Gui-Lin, Liu Shao-Hui, Li Min, Yao Hong-Xun, Hu Wen. Applications and analyses on post processing in image matting. Acta Electronica Sinica, 2017, 45(3): 719-729 doi: 10.3969/j.issn.0372-2112.2017.03.032 [28] 张展鹏, 朱青松, 谢耀钦.数字抠像的最新研究进展. 自动化学报, 2012, 38(10): 1571-1584 doi: 10.3724/SP.J.1004.2012.01571Zhang Zhan-Peng, Zhu Qing-Song, Xie Yao-Qin. The latest research progress on digital matting. Acta Automatica Sinica, 2012, 38(10): 1571-1584 doi: 10.3724/SP.J.1004.2012.01571 [29] Zhou Y, He F Z, Hou N, Qiu Y M. Parallel ant colony optimization on multi-core SIMD CPUs. Future Generation Computer Systems, 2018, 79: 473-487 doi: 10.1016/j.future.2017.09.073 [30] Zhou Y, He F Z, Qiu Y M. Dynamic strategy based parallel ant colony optimization on GPUs for TSPs. Science China Information Sciences, 2017, 60(6): 260-262 -
点击查看大图
计量
- 文章访问数: 783
- HTML全文浏览量: 141
- PDF下载量: 128
- 被引次数: 0
