2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

自适应分割的视频点云多模式帧间编码方法

陈建 廖燕俊 王适 郑明魁 苏立超

陈建, 廖燕俊, 王适, 郑明魁, 苏立超. 自适应分割的视频点云多模式帧间编码方法. 自动化学报, 2023, 49(8): 1707−1722 doi: 10.16383/j.aas.c220549
引用本文: 陈建, 廖燕俊, 王适, 郑明魁, 苏立超. 自适应分割的视频点云多模式帧间编码方法. 自动化学报, 2023, 49(8): 1707−1722 doi: 10.16383/j.aas.c220549
Chen Jian, Liao Yan-Jun, Wang Kuo, Zheng Ming-Kui, Su Li-Chao. An adaptive segmentation based multi-mode inter-frame coding method for video point cloud. Acta Automatica Sinica, 2023, 49(8): 1707−1722 doi: 10.16383/j.aas.c220549
Citation: Chen Jian, Liao Yan-Jun, Wang Kuo, Zheng Ming-Kui, Su Li-Chao. An adaptive segmentation based multi-mode inter-frame coding method for video point cloud. Acta Automatica Sinica, 2023, 49(8): 1707−1722 doi: 10.16383/j.aas.c220549

自适应分割的视频点云多模式帧间编码方法

doi: 10.16383/j.aas.c220549
基金项目: 国家自然科学基金(62001117, 61902071), 福建省自然科学基金(2020J01466), 中国福建光电信息科学与技术创新实验室(闽都创新实验室) (2021ZR151), 超低延时视频编码芯片及其产业化(2020年福建省教育厅产学研专项)资助
详细信息
    作者简介:

    陈建:福州大学物理与信息工程学院副教授. 主要研究方向为视频编码, 压缩感知, 点云压缩和目标跟踪. E-mail: chenjian-fzu@163.com

    廖燕俊:福州大学先进制造学院硕士研究生. 主要研究方向为点云分割和视频点云压缩. E-mail: liao.yanjun@foxmail.com

    王适:福州大学物理与信息工程学院硕士研究生. 主要研究方向为多媒体技术. E-mail: wang_kuo@cib.com.cn

    郑明魁:福州大学物理与信息工程学院副教授. 主要研究方向为计算机视觉, 点云与视频编码. 本文通信作者. E-mail: zhengmk@fzu.edu.cn

    苏立超:福州大学计算机与大数据学院/软件学院讲师. 主要研究方向为多媒体信息安全. E-mail: fzu-slc@fzu.edu.cn

An Adaptive Segmentation Based Multi-mode Inter-frame Coding Method for Video Point Cloud

Funds: Supported by National Natural Science Foundation of China (62001117, 61902071), Fujian Natural Science Foundation (2020J01466), Fujian Science & Technology Innovation Laboratory for Optoelectronic Information of China (2021ZR151), and Ultra-low Latency Video Coding Chip and its Industrialization (2020 Special Project of Fujian Provincial Education Department for Industry-University Research)
More Information
    Author Bio:

    CHEN Jian Associate professor at the College of Physics and Information Engineering, Fuzhou University. Her research interest covers video coding, compression sensing, point cloud compression and target tracking

    LIAO Yan-Jun Master student at the School of Advanced Manufacturing, Fuzhou University. His research interest covers point cloud segmentation and dynamic point cloud compression

    WANG Kuo Master student at the College of Physics and Information Engineering, Fuzhou University. His main research interest is multimedia technology

    ZHENG Ming-Kui Associate professor at the College of Physics and Information Engineering, Fuzhou University. His research interest covers computer vision, video and point cloud coding. Corresponding author of this paper

    SU Li-Chao Lecturer at the College of Computer and Data Science/College of Software, Fuzhou University. His main research interest is multimedia infomation security

  • 摘要: 基于视频的点云压缩(Video based point cloud compression, V-PCC)为压缩动态点云提供了高效的解决方案, 但V-PCC从三维到二维的投影使得三维帧间运动的相关性被破坏, 降低了帧间编码性能. 针对这一问题, 提出一种基于V-PCC改进的自适应分割的视频点云多模式帧间编码方法, 并依此设计了一种新型动态点云帧间编码框架. 首先, 为实现更精准的块预测, 提出区域自适应分割的块匹配方法以寻找最佳匹配块; 其次, 为进一步提高帧间编码性能, 提出基于联合属性率失真优化(Rate distortion optimization, RDO)的多模式帧间编码方法, 以更好地提高预测精度和降低码率消耗. 实验结果表明, 提出的改进算法相较于V-PCC实现了−22.57%的BD-BR (Bjontegaard delta bit rate)增益. 该算法特别适用于视频监控和视频会议等帧间变化不大的动态点云场景.
  • 图  1  V-PCC编码器框架

    Fig.  1  V-PCC encoder diagram

    图  2  V-PCC从三维到二维投影(属性图)

    Fig.  2  V-PCC projection from 3D to 2D (Attribute map)

    图  3  改进的三维帧间编码框架

    Fig.  3  Improved 3D inter-frame coding framework

    图  4  区域自适应分割块匹配方法示意图

    Fig.  4  Schematic diagram of region adaptive segmentation based block matching method

    图  5  区域自适应分割的块匹配方法分割示例

    Fig.  5  Example of block matching method based on adaptive regional segmentation

    图  6  率失真折线与$\lambda$示意图

    Fig.  6  Rate distortion polyline and $\lambda$ diagram

    图  7  8i动态点云序列示例

    Fig.  7  Example of 8i dynamic point cloud sequence

    图  8  消融实验率失真曲线图

    Fig.  8  Rate distortion curve of ablation experiment

    图  9  Soldier序列率失真曲线图

    Fig.  9  Rate distortion curve of Soldier sequence

    表  1  相对近似块帧间编码比特占用

    Table  1  RSB inter coding bits occupancy

    编码类型 编码信息 数据类型 比特占用(bits)
    位置信息 $(X,Y,Z)_{{\rm{min}},{\rm{max}}}$ Int 96
    旋转矩阵 $3\times3$矩阵 Float 288
    平移向量 $3\times 1$向量 Float 96
    属性偏移 $(R,G,B)$ Int 48
    总编码消耗 $R_{\rm{RSB}}$ Int、Float 528
    下载: 导出CSV

    表  2  提出的改进算法量化参数

    Table  2  Quantization parameters of proposed improved algorithm

    量化参数 编码模式判定$\Omega$ 最小块阈值$N_D$ 分割截止阈值$T$
    R1 3.5 14000 2.0
    R2 3.0 12000 2.2
    R3 2.5 10000 2.5
    R4 2.0 8000 2.8
    R5 1.5 6000 3.0
    下载: 导出CSV

    表  3  区域自适应分割的块匹配方法性能测试

    Table  3  Performance test of block matching method based on adaptive regional segmentation

    Name D1 (%) D2 (%) Luma (%) Cb (%) Cr (%)
    均匀块分割 Loot 33.93 49.67 4.48 −18.07 −15.08
    Redandblack 13.51 19.42 13.41 −3.01 10.79
    Soldier −34.75 −34.66 −40.82 −42.91 −44.52
    Queen −33.64 −33.33 −18.33 −28.58 −12.74
    Longdress 11.97 13.15 27.48 −4.88 11.13
    Average −1.80 −2.85 −2.76 −18.51 −10.08
    改进的自适应块分割 Loot 27.10 39.28 2.22 −24.14 −21.19
    Redandblack 6.03 8.14 16.06 4.14 13.72
    Soldier −37.69 −37.57 −42.87 −45.10 −46.42
    Queen −35.72 −33.94 −9.36 −12.93 −14.92
    Longdress 6.74 6.46 4.83 5.02 12.49
    Average −6.71 −3.53 −5.82 −14.60 −11.26
    相对BD-BR增益 −4.91 −6.38 −3.06 3.91 −1.18
    下载: 导出CSV

    表  4  联合属性率失真优化的多模式帧间编码性能测试

    Table  4  Performance test of multi-mode inter-frame coding based on joint attribute rate distortion optimization

    Name D1 (%) D2 (%) Luma (%) Cb (%) Cr (%)
    完全帧间运动预测 Loot 32.31 42.75 11.16 −19.63 −16.09
    Redandblack 14.16 15.52 7.88 −11.11 6.35
    Soldier −31.21 −31.21 −37.47 −41.07 −42.48
    Queen −29.34 −30.23 −12.47 −17.84 −6.77
    Longdress 12.28 14.79 17.74 −13.26 0.79
    Average −0.36 2.32 −2.63 −20.58 −13.70
    改进的多模式帧间运动预测 Loot −0.06 0.70 10.07 5.44 5.80
    Redandblack −3.31 −2.09 11.73 1.66 9.45
    Soldier −35.88 −34.59 −36.31 −36.28 −38.70
    Queen −39.55 −39.37 −39.16 −44.46 −44.97
    Longdress −0.64 1.28 6.50 3.10 10.23
    Average −18.13 −17.09 −9.43 −16.06 −11.64
    相对BD-BR增益 −17.77 −19.41 −6.80 4.52 2.06
    下载: 导出CSV

    表  5  相较于V-PCC相互消融性能(平均BD-BR)

    Table  5  Mutual ablation performance compared to V-PCC (Average BD-BR)

    应用算法 D1 (%) D2 (%) Luma (%) Cb (%) Cr (%)
    多模式帧间编码 −18.13 −17.09 −9.43 −16.06 −11.64
    自适应块分割 −6.71 −3.53 −5.82 −14.60 −11.26
    完整算法框架 −22.57 −20.94 −22.01 −23.67 −21.90
    下载: 导出CSV

    表  6  提出的改进算法相较于V-PCC的性能比较

    Table  6  Performance of proposed improved algorithm compared with V-PCC

    Name D1 (%) D2 (%) Luma (%) Cb (%) Cr (%)
    Loot −2.64 −0.69 −3.57 −4.74 −4.85
    Redandblack −1.65 −0.95 −1.75 −2.01 −2.12
    Soldier −51.53 −46.99 −54.12 −55.65 −55.95
    Queen −56.58 −56.45 −51.62 −55.12 −46.27
    Longdress −0.44 0.40 0.99 −0.80 −0.30
    Overall −22.57 −20.94 −22.01 −23.67 −21.90
    下载: 导出CSV

    表  7  提出的改进算法同文献[15, 21-22]的性能对比

    Table  7  Performance comparisons between the proposed improved algorithm and references [15, 21-22]

    Name D1 (%) D2 (%) Luma (%) Cb (%) Cr (%)
    文献[15] −21.02 −15.68 −3.54
    文献[21] −4.70 −4.70 −9.70 −16.30 −14.50
    文献[22] −14.80 −19.10 −2.20 −16.20 −19.90
    提出的算法 −22.57 −20.94 −22.01 −23.67 −21.90
    下载: 导出CSV

    表  8  相较于V-PCC的编码开销与时间复杂度

    Table  8  Encoding overhead and time complexity compared to V-PCC

    量化
    等级
    预测
    编码
    (bits)
    NSB
    编码
    (bits)
    V-PCC
    帧内
    编码(bits)
    相对编码
    开销(%)
    相对编码
    时间(%)
    Queen R1 5722 33065 97363 39.83 85.01
    R2 6099 48154 140968 38.49 84.74
    R3 6841 78773 225197 38.02 84.99
    R4 6797 149308 392872 39.71 85.77
    R5 7494 292382 686758 43.67 92.08
    Soldier R1 4986 47469 149301 35.13 82.17
    R2 5692 76517 236022 34.83 81.79
    R3 7986 138810 392283 37.42 81.27
    R4 11836 305130 675218 46.94 83.38
    R5 16182 826968 1192585 70.69 94.29
    Average 7955 199658 418857 49.56 85.55
    下载: 导出CSV
  • [1] 李厚强, 李礼, 李竹. 点云编码综述. 中兴通讯技术, 2021, 27(01):5−9

    Li Hou-Qiang, Li Li, Li Zhu. A review of point cloud compression. ZTE Technology Journal, 2021, 27(01):5−9
    [2] Ishikawa Y, Hachiuma R, Ienaga N, Kuno W, Sugiura Y, Saito H. Semantic segmentation of 3D point cloud to virtually manipulate real living space. In: Proceedings of the Asia Pacific Workshop on Mixed and Augmented Reality. Ikoma, Japan: IEEE, 2019. 1−7
    [3] Deng X M, Zhu Y Y, Zhang Y D, Cui Z P, Tan P, Qu W, et al. Weakly supervised learning for single depth-based hand shape recovery. IEEE Transactions on Image Processing, 2021, 30: 532−545 doi: 10.1109/TIP.2020.3037479
    [4] 田永林, 沈宇, 李强, 王飞跃. 平行点云:虚实互动的点云生成与三维模型进化方法. 自动化学报, 2020, 46(12): 2572−2582 doi: 10.16383/j.aas.c200800

    Tian Yong-Lin, Shen Yu, Li Qiang, Wang Fei-Yue. Parallel point clouds: point clouds generation and 3D model evolution via virtual-real interaction. Acta Automatica Sinica, 2020, 46(12): 2572−2582 doi: 10.16383/j.aas.c200800
    [5] Sun P P, Zhao X M, Xu Z G, Wang R M, Min H G. A 3D LiDAR data-based dedicated road boundary detection algorithm for autonomous vehicles. IEEE Access, 2019, 7: 29623−29638 doi: 10.1109/ACCESS.2019.2902170
    [6] Huang X Y, Wang P, Cheng X J, Zhou D F, Geng Q C, Yang R G. The apolloscape open dataset for autonomous driving and its application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(10): 2702−2719 doi: 10.1109/TPAMI.2019.2926463
    [7] Gan Z P, Xu H R, He Y R, Cao W, Chen G H. Autonomous landing point retrieval algorithm for UAVs based on 3D environment perception. In: Proceedings of the IEEE 7th International Conference on Virtual Reality. Foshan, China: IEEE, 2021. 104−108
    [8] 曹成坤, 张琮毅, 汪国平. 精度可控的字典全局相似性点云压缩. 计算机辅助设计与图形学学报, 2019, 31(6):869−877

    Cao Cheng-Kun, Zhang Cong-Yi, Wang Guo-Ping. Precision controllable point clouds compression using global similarity in dictionary. Journal of Computer-Aided Design & Computer Graphics, 2019, 31(6): 869−877
    [9] Maja K, Chou P A, Savill P. 8i voxelized surface light field (8iVSLF) dataset. ISO/IEC JTC1/SC29/WG11 MPEG, input document m42914, Ljubljana, Slovenia, 2018
    [10] Kammerl J, Blodow N, Rusu R B, Gedikli S, Beetz M, Steinbach E. Real-time compression of point cloud streams. In: Proceedings of the IEEE International Conference on Robotics and Automation. Saint Paul, USA: IEEE, 2012. 778−785
    [11] Thanou D, Chou P A, Frossard P. Graph-based compressionof dynamic 3D point cloud sequences. IEEE Transactions on Image Processing, 2016, 25(4): 1765−1778 doi: 10.1109/TIP.2016.2529506
    [12] Queiroz R L, Chou P A. Motion-compensated compression of dynamic voxelized point clouds. IEEE Transactions on Image Processing, 2017, 26(8): 3886−3895 doi: 10.1109/TIP.2017.2707807
    [13] Mekuria R, Blom K, Cesar P. Design, implementation, and evaluation of a point cloud codec for tele-immersive video. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(4): 828−842 doi: 10.1109/TCSVT.2016.2543039
    [14] Besl P J, McKay N D. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(2):239−256 doi: 10.1109/34.121791
    [15] Santos C, Goncalves M, Correa G, Porto M. Block-based inter-frame prediction for dynamic point cloud compression. In: Proceedings of the IEEE International Conference on Image Processing. Anchorage, USA: IEEE, 2021. 3388−3392
    [16] Lasserre S, Llach J, Guede C, Ricard J. Technicolor's response to the CfP for point cloud compression. ISO/IEC JTC1/SC29/WG11 MPEG, input document m41822, Macau, China, 2017
    [17] Budagavi M, Faramarzi E, Ho T, Najaf-Zadeh H, Sinharoy I. Samsung's response to CfP for point cloud compression (Category2). ISO/IEC JTC1/SC29/WG11 MPEG, input document m41808, Macau, China, 2017
    [18] Schwarz S, Sheikhipour N, Sevom V F, Hannuksela M M. Video coding of dynamic 3D point cloud data. APSIPA Transactions on Signal and Information Processing, DOI: 10.1017/ATSIP.2019.24
    [19] Mammou K, Tourapis A M, SINGER D. Video-based and hierarchical approaches point cloud compression. ISO/IEC JTC1/SC29/WG11 MPEG, input document m41779, Macua, China, 2017
    [20] Graziosi D, Nakagami O, Kuma S, Zaghetto A, Suzuki T, Tabatabai A. An overview of ongoing point cloud compression standardization activities: Video-based (V-PCC) and geometry-based (G-PCC). APSIPA Transactions on Signal and Information Processing, DOI: 10.1017/ATSIP.2020.12
    [21] Li L, Zhu L, Zakharchenko V, Chen J L, Li H Q. Advanced 3D motion prediction for Video-based dynamic point cloud compression. IEEE Transactions on Image Processing, 2020, 29:289−302 doi: 10.1109/TIP.2019.2931621
    [22] Kim J, Im J, Rhyu S, Kim K. 3D motion estimation and compensation method for Video-based point cloud compression. IEEE Access, 2020, 8:83538−83547 doi: 10.1109/ACCESS.2020.2991478
    [23] Costa A, Dricot A, Brites C, Ascenso J, Pereira F. Improved patch packing for the MPEG V-PCC standard. In: Proceedings of the IEEE 21st International Workshop on Multimedia Signal Processing. Kuala Lumpur, Malaysia: IEEE, 2019. 1−6
    [24] Li L, Li Z, Liu S, Li H Q. Occupancy-map-based rate distortion optimization and partition for video-based point cloud compression. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 31(1): 326−338
    [25] Li L, Li Z, Liu S, Li H Q. Efficient projected frame padding for video-based point cloud compression. IEEE Transactions on Multimedia, 2021, 23: 2806−2819 doi: 10.1109/TMM.2020.3016894
    [26] Xu Y Q, Hu W, Wang S S, Zhang X F, et al. Predictive generalized graph fourier transform for attribute compression of dynamic point clouds. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(5): 1968−1982 doi: 10.1109/TCSVT.2020.3015901
    [27] Schwarz S, Martin-Cocher G, Flynn D, Budagavi M. Common test conditions for point cloud compression. ISO/IEC JTC1/SC29/WG11 MPEG, input docment w17766, Ljubljana, Slovenia, 2020
    [28] Jang E S, Preda M, Mammou K, Tourapis A M, et al. Video-based point-cloud-compression standard in MPEG: from evidence collection to committee draft. IEEE Signal Processing Magazine, 2019, 36(3): 118−123 doi: 10.1109/MSP.2019.2900721
  • 加载中
图(9) / 表(8)
计量
  • 文章访问数:  429
  • HTML全文浏览量:  138
  • PDF下载量:  133
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-07-05
  • 录用日期:  2022-11-29
  • 网络出版日期:  2023-07-06
  • 刊出日期:  2023-08-21

目录

    /

    返回文章
    返回