2.845

2023影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

多视点稀疏测量的图像绘制方法

兰诚栋 林宇鹏 方大锐 陈建

兰诚栋, 林宇鹏, 方大锐, 陈建. 多视点稀疏测量的图像绘制方法.自动化学报, 2021, 47(4): 882-890 doi: 10.16383/j.aas.c180199
引用本文: 兰诚栋, 林宇鹏, 方大锐, 陈建. 多视点稀疏测量的图像绘制方法.自动化学报, 2021, 47(4): 882-890 doi: 10.16383/j.aas.c180199
Lan Cheng-Dong, Lin Yu-Peng, Fang Da-Rui, Chen Jian. Multi-view sparse measurement for image-based rendering method. Acta Automatica Sinica, 2021, 47(4): 882-890 doi: 10.16383/j.aas.c180199
Citation: Lan Cheng-Dong, Lin Yu-Peng, Fang Da-Rui, Chen Jian. Multi-view sparse measurement for image-based rendering method. Acta Automatica Sinica, 2021, 47(4): 882-890 doi: 10.16383/j.aas.c180199

多视点稀疏测量的图像绘制方法

doi: 10.16383/j.aas.c180199
基金项目: 

国家自然科学基金 62001117

国家自然科学基金 61671152

国家自然科学基金 61471124

福建省自然科学基金 2017J01757

福建省自然科学基金 2014J01234

福州大学科研启动基金 GXRC-17034

详细信息
    作者简介:

    林宇鹏  福州大学物理与信息工程学院硕士研究生. 主要研究方向为图像处理. E-mail: 18850766818@163.com

    方大锐  福州大学物理与信息工程学院硕士研究生. 主要研究方向为图像处理与压缩感知应用. E-mail: N161120076@fzu.edu.cn

    陈建  福州大学物理与信息工程学院讲师. 主要研究方向为视频编码, 压缩感知, 点云压缩. E-mail: chenjian-fzu@163.com

    通讯作者:

    兰诚栋  福州大学物理与信息工程学院副教授. 主要研究方向为图像处理与视频编码. 本文通信作者. E-mail: lancd@fzu.edu.cn

Multi-view Sparse Measurement for Image-based Rendering Method

Funds: 

National Natural Science Foundation of China 62001117

National Natural Science Foundation of China 61671152

National Natural Science Foundation of China 61471124

Natural Science Foundation of FuJian Province 2017J01757

Natural Science Foundation of FuJian Province 2014J01234

Fuzhou University Research Startup Fund GXRC-17034

More Information
    Author Bio:

    LIN Yu-Peng  Master student at the College of Physics and Information Engineering, Fuzhou University. His main research interest is image processing

    FANG Da-Rui  Master student at the College of Physics and Information Engineering, Fuzhou University. His research interest covers image processing and the application of compressed sensing

    CHEN Jian  Lecturer at the College of Physics and Information Engineering, Fuzhou University. Her research interest covers video coding, compressed sensing, and point cloud compression

    Corresponding author: LAN Cheng-Dong  Associate professor at the College of Physics and Information Engineering, Fuzhou University. His research interest covers image processing and video coding. Corresponding author of this paper
  • 摘要: 为了减少所需采集的视频数据量, 基于图像绘制(Image-based rendering, IBR) 的前沿方法将稠密视点信息映射成压缩感知框架中的原始信号, 并将稀疏视点图像作为随机测量值, 但低维测量信号由所有稠密视点信息线性组合而成, 而稀疏视点图像仅仅来源于部分视点信息, 导致稀疏视点采集的图像与低维测量信号不一致. 本文提出利用间隔采样矩阵消除测量信号与稀疏视点图像位置之间的差异, 进而通过约束由测量矩阵和基函数构成的传感矩阵尽量满足有限等距性, 使得能够获得原始信号的唯一精确解. 仿真实验结果表明, 相比于前沿方法, 本文提出的方法对于不同复杂程度的场景重建都提高了主客观质量.
    Recommended by Associate Editor WU Yi-Hong
    1)  本文责任编委 吴毅红
  • 图  1  基于压缩感知框架的稠密多视点图像重建原理

    Fig.  1  Principle of dense multi-view image reconstruction based on compressed sensing framework

    图  2  构建稀疏视点测量矩阵与自适应稀疏基函数

    Fig.  2  Constructing sparse viewpoint measurement matrix and adaptive sparse basis function

    图  3  EPI及其频谱示意图

    Fig.  3  EPI and its spectrum diagram

    图  4  图像绘制算法流程

    Fig.  4  Image rendering algorithm flow

    图  5  基向量个数与重建误差的关系

    Fig.  5  The relationship between the number of base vectors and reconstruction error

    图  6  主观质量对比图((a)原始图像; (b)傅里叶频域滤波; (c)小波基稀疏重建; (d)本文重建方法)

    Fig.  6  Subjective quality comparison chart ((a) Original image; (b) Fourier frequency domain filtering; (c) Wavelet base sparse reconstruction; (d) Reconstruction method)

    表  1  算法参数说明

    Table  1  Algorithm parameter description

    重建方法 测试序列 压缩传感矩阵 采样点倍数 测量矩阵
    傅里叶频域滤波重建 8组斯坦福公共测试序列 基于傅里叶基 0.5 多视点间隔测量矩阵
    小波基稀疏重建 8组斯坦福公共测试序列 基于小波基 0.5 多视点间隔测量矩阵
    多视点稀疏测量约束重建 8组斯坦福公共测试序列 基于多视点稀疏测量约束 0.5 多视点间隔测量矩阵
    下载: 导出CSV

    表  2  重建图像客观质量PSNR (平均值)比较

    Table  2  Comparison of objective quality PSNR (average) of reconstructed images

    重建方法 测试序列
    Bracelet Bunny Cards and ball Chess Jelly Beans Knights Bulldozer Truck
    傅里叶频域滤波重建 0.84 0.93 0.80 0.92 0.95 0.85 0.78 0.91
    小波基稀疏重建 0.95 0.81 0.94 0.95 0.98 0.94 0.92 0.96
    多视点稀疏测量约束重建 0.97 0.94 0.97 0.96 0.96 0.98 0.95 0.94
    下载: 导出CSV

    表  3  重建图像客观质量SSIM (平均值)比较

    Table  3  Comparison of objective quality SSIM (average) of reconstructed images

    重建方法 测试序列
    Bracelet Bunny Cards and ball Chess Jelly Beans Knights Bulldozer Truck
    傅里叶频域滤波重建 23.06 34.22 22.15 30.44 34.33 25.01 23.66 33.13
    小波基稀疏重建 30.15 36.56 30.33 34.61 39.30 32.64 31.34 40.76
    多视点稀疏测量约束重建 37.39 39.63 36..35 39.21 38.40 37.34 41.32 40.29
    下载: 导出CSV
  • [1] Niamut O A, Kochale A, Hidalgo J R, Kaiser R, Spille J, Macq J F, et al. Towards a format-agnostic approach for production, delivery and rendering of immersive media. In: Proceedings of the 4th ACM Multimedia Systems Conference. Oslo, Norway: ACM, 2013. 249-260
    [2] Deng C Y, Zhou Z G, Li W Q, Hou B Y. A panoramic geology field trip system using image-based rendering. In: Proceedings of the 40th IEEE Computer Software and Applications Conference. Atlanta, Georgia: IEEE, 2016. 264-268
    [3] Doumanoglou A, Griffin D, Serrano J, Zioulis N. Quality of experience for 3-D immersive media streaming. IEEE Transactions on Broadcasting, 2018, 64(2): 379-391 doi: 10.1109/TBC.2018.2823909
    [4] Zollhofer M, Stotko P, Gorlitz A, Theobalt C, Niessner M, Klein R, et al. State of the art on 3D reconstruction with RGB-D cameras. Computer Graphics Forum, 2018, 37(2): 625-652 doi: 10.1111/cgf.13386
    [5] Kim C, Zimmer H, Pritch Y, Sorkine-Hornung A, Gross M. Scene reconstruction from high spatio-angular resolution light fields. ACM Transactions on Graphics, 2013, 32(4): 1-12
    [6] 刘伟, 吴毅红. 基于图层优化与融合的2D-3D视频转换方法. 计算机辅助设计与图形学学报. 2012, 24(11): 1426-1439 doi: 10.3969/j.issn.1003-9775.2012.11.007

    Liu Wei, Wu Yi-Hong. A 2D-3D video conversion method based on layer optimization and integration. Journal of Computer-Aided Design & Computer Graphics, 2012, 24(11): 1426-1439 doi: 10.3969/j.issn.1003-9775.2012.11.007
    [7] Liu W, Wu Y H, Guo F S, Hu Z Y. An efficient approach of 2D to 3D video conversion based on piece-wise structure from motion. The Visual Computer, 2015, 31(1): 55-68 doi: 10.1007/s00371-013-0904-3
    [8] Wu G C, Liu Y B, Fang L, Dai Q H, Chai T Y. Light field reconstruction using convolutional network on EPI and extended applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(1): 1681-1694 doi: 10.1109/TPAMI.2018.2845393
    [9] Zhu H, Wang Q, Yu J Y. Light field imaging: models, calibrations, reconstructions, and applications. Frontiers of Information Technology & Electronic Engineering, 2017, 18(9): 1236-1249
    [10] Chai J X, Tong X, Chan S C, Shum H Y. Plenoptic sampling. In: Proceedings of the 27th annual conference on computer graphics and interactive techniques, New York, USA: ACM, 2000. 307-318
    [11] Zhang C, Chen T. Spectral analysis for sampling image-based rendering data. IEEE Transactions on Circuits and Systems for Video Technology, 2003, 13(11): 1038-1050 doi: 10.1109/TCSVT.2003.817350
    [12] Do M N, Marchand-Maillet D, Vetterli M. On the bandlimitedness of the plenoptic function. In: Processings of the 2005 IEEE International Conference on Image Processing, New York, USA: IEEE, 2005. 17-20
    [13] Do M N, Marchand-Maillet D, Vetterli M. On the bandwidth of the plenoptic function. IEEE Transactions on Image Processing, 2012, 21(2): 708-717 doi: 10.1109/TIP.2011.2163895
    [14] Zhou P, Yu L, Zhong G. The non-lambertian reflection in plenoptic sampling. In: Proceedings of the 2013 IEEE International Conference on Image Processing. New York, USA: IEEE, 2013. 2154-2157
    [15] Gilliam C, Dragotti P L, Brookes M. On the spectrum of the plenoptic function. IEEE Transactions on Image Processing, 2014, 23(2): 502-516 doi: 10.1109/TIP.2013.2292363
    [16] Vagharshakyan S, Bregovic R, Gotchev A. Image based rendering technique via sparse representation in shearlet domain. In: Prceedings of International Conference on Image Processing. New York, USA: IEEE, 2015. 1379-1383
    [17] Heber S, Pock T. Convolutional networks for shape for light field. In: Proceedings of the 2016 IEEE International Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE, 2016. 3746-3754
    [18] Kalantari N K, Wang T C, Ramamoorthi R. Learning based view synthesis for light field cameras. ACM Transactions on Graphics, 2016, 35(6): 1-10
    [19] Unde A S, Deepthi P P. Block compressive sensing: Individual and joint reconstruction of correlated images. Journal of Visual Communication and Image Representation, 2017, 44: 187-197 doi: 10.1016/j.jvcir.2017.01.028
    [20] Ansari N, Gupta A. Image reconstruction using matched wavelet estimated from data sensed compressively using partial canonical identity matrix. IEEE Transaction on Image Processing, 2017, 26(8): 3680-3695 doi: 10.1109/TIP.2017.2700719
    [21] Vagharshakyan S, Bregovic R, Gotchev A. Light field reconstruction using shearlet transform. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(1): 133-147 doi: 10.1109/TPAMI.2017.2653101
    [22] Ji S H, Xue Y. Carin L. Bayesian compressive sensing. IEEE Transactions on Signal Processing, 2008, 56(6): 2346-2356 doi: 10.1109/TSP.2007.914345
    [23] Candes E J, Romberg J, Tao T. Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics, 2006, 59(8): 1207-1223 doi: 10.1002/cpa.20124
    [24] Candes E J, Romberg J, Tao T. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory, 2006, 52(2): 489-509 doi: 10.1109/TIT.2005.862083
    [25] Bolles R C, Baker H H, Marimont D H. Epipolar-plane image analysis: an approach to determining structure from motion. International Journal of Computer Vision, 1987, 1(1): 7-55 doi: 10.1007/BF00128525
    [26] 陈允杰, 葛魏东, 孙乐. 一种基于协同稀疏和全变差的高光谱线性解混方法. 自动化学报, 2017, 44(1): 116-128 doi: 10.16383/j.aas.2018.c160414

    Chen Yun-Jie, Ge Wei-Dong, Sun Le. A novel linear hyperspectral unmixing method based on collaborative sparsity and total variation. Acta Automatica Sinica, 2017, 44(1): 116-128 doi: 10.16383/j.aas.2018.c160414
    [27] 常振春, 禹晶, 肖创柏, 孙卫东. 基于稀疏表示和结构自相似性的单幅图像盲解卷积算法. 自动化学报, 2017, 43(11): 1908-1919 doi: 10.16383/j.aas.2017.c160357

    Chang Zhen-Chun, Yu Jing, Xiao Chuang-Bai, Sun Wei-Dong. Single image blind deconvolution using sparse representation and structural self-similarity. Acta Automatica Sinica, 2017, 43(11): 1908-1919 doi: 10.16383/j.aas.2017.c160357
    [28] Vaish V, Adams A. The (New) Stanford light field archive [Online], available: http://lightfield.stanford.edu, October 18, 2018
    [29] 黄博学, 周彤. 利用Block-StOMP的一种改进算法高效重构块稀疏信号. 自动化学报, 2017, 43(9): 1607-1618 doi: 10.16383/j.aas.2017.e150116

    Huang B X, Zhou T. Efficient recovery of block sparse signals by an improved algorithm of Block-StOMP. Acta Automatica Sinica, 2017, 43(9): 1607-1618 doi: 10.16383/j.aas.2017.e150116
  • 加载中
图(6) / 表(3)
计量
  • 文章访问数:  840
  • HTML全文浏览量:  137
  • PDF下载量:  124
  • 被引次数: 0
出版历程
  • 收稿日期:  2018-04-09
  • 录用日期:  2018-10-11
  • 刊出日期:  2021-04-23

目录

    /

    返回文章
    返回