-
摘要: 由于点云的非结构性和无序性, 目前已有的点云分类网络在精度上仍然需要进一步提高. 通过考虑局部结构的构建、全局特征聚合和损失函数改进三个方面, 本文构造了一个有效的点云分类网络. 首先, 针对点云的非结构性,通过学习中心点特征与近邻点特征之间的关系, 为不规则的近邻点分配不同的权重, 以此构建局部结构. 此外,使用注意力的思想, 提出了加权平均池化, 通过自注意力的方式, 学习每个高维特征的注意力分数, 在应对点云无序性的同时, 可以有效地聚合冗余的高维特征. 另外,利用了交叉熵损失与中心损失之间的互补关系, 提出了联合损失, 在增大类间距离的同时减小了类内距离, 进一步提高了网络的分类能力. 本文在合成数据集ModelNet40、ShapeNetCore和真实世界数据集ScanObjectNN上进行了实验, 与目前性能最好的多个网络相比较, 验证了本文整体网络结构的优越性.Abstract: Due to the unstructured and disordered nature of point cloud, the classification accuracy is still needed to improve for the currently existed point cloud classification approaches. In this paper, an effective point cloud classification network is proposed by considering local structure construction, global feature aggregation and loss function improvement. Firstly, in view of the unstructured nature of point cloud, the local structure is constructed via assigning different weights to those irregular neighborhood points, which are computed via the relationship between the center point feature and its neighbor feature. Moreover, a strategy of weighted average pooling is designed to learn the attention score of each high-dimensional feature through self-attention mechanization, which can effectively aggregate redundant high-dimensional features while coping with point cloud disorder. In addition, a joint loss function is presented by utilizing the complementary relationship between the cross-entropy loss and the center loss, which increases the inter-class distance while reduces the intra-class distance, and further improves the classification ability of the network. Compared to several state-of-the-art networks, the experimental results on the synthetic dataset ModelNet40 and ShapeNetCore and the real-world dataset ScanObjectNN demonstrate the superiority of the overall network structure.
-
Key words:
- Deep learning /
- 3D point cloud /
- point cloud classification /
- attention mechanism /
- loss function
-
图 2 中心点和近邻点的局部结构示意图
Fig. 2 Diagram of local structure between the central point and its neighbors
图 7 采样密度实验结果对比
Fig. 7 Comparison of experiment results with different sampling densities
图 8 添加高斯噪声前后点云样本的对照图
Fig. 8 A comparison of one sample with and without Gaussian noise
表 1 实验配置
Table 1 Experimental configuration
项目 详情 操作系统 CentOS Linux 7 GPU Tesla V100 32G CUDA 10.0 CUDNN 7.0 Python 3.6.0 Pytorch 1.3.0 
下载: 导出CSV
表 2 在ModelNet40数据集上的实验结果
Table 2 Experimental results on ModelNet40
Method Input mAcc(%) OA(%) PointNet [3] 1k 86.0 89.2 PointNet++[10] 1k - 90.7 Spec-GCN[11] 1k - 91.5 DGCNN[12] 1k 90.2 92.9 RSCNN[13] 1k - 92.9 PCT[16] 1k - 93.2 ECC[21] 1k 83.2 87.4 RMFP-DNN[22] 1k 88.9 92.6 PointCNN[23] 1k 88.1 92.2 P2Sequence[24] 1k 90.4 92.6 Point Transformer [25] 1k - 92.8 Octant-CNN[26] 1k 88.7 91.9 DRNet[27] 1k - 93.1 AdaptConv[28] 1k 90.7 93.4 RFNet 1k 91.2 93.6 PointNet++[10] 5k+nor - 91.9 DGCNN[12] 2k 90.7 93.5 RSCNN[13] 1k+voting - 93.6 SpiderCNN[29] 5k+nor - 92.4 RFNet 2k 91.4 94.0
下载: 导出CSV
表 3 不同网络的易错模型分类对比
Table 3 Classification comparison of error-prone models for different networks
下载: 导出CSV
表 5 在ScanObjectNN数据集上的实验结果
Table 5 Experimental results on ScanObjectNN
Method mAcc (%) OA (%) Bag Bin Box Sofa Desk Shelf table Door Bed Cabinet Chair Display Pillow Sink Toilet PointNet[3] 63.4 68.2 36.1 69.8 10.5 76.7 50.0 72.6 67.8 93.8 61.8 62.6 89.0 73.0 67.6 64.2 55.3 PointNet++[10] 75.4 77.9 49.4 84.4 31.6 90.5 74.0 72.6 72.6 85.2 75.5 77.4 91.3 79.4 81.0 80.8 85.9 DGCNN[12] 73.6 78.1 49.4 82.4 33.1 91.4 63.3 79.3 77.4 89.0 64.5 83.9 91.8 77.0 77.1 75.0 69.4 PointCNN[23] 75.1 78.5 57.8 82.9 33.1 91.9 65.3 84.2 67.4 84.8 80.0 83.6 92.6 78.4 80.0 72.5 71.8 SpiderCNN [29] 69.8 73.7 43.4 75.9 12.8 90.5 65.3 78.0 65.9 91.4 69.1 74.2 89.0 74.5 80.0 65.8 70.6 RFNet 76.3 79.6 54.0 81.9 34.1 92.2 74.1 79.9 72.1 91.5 81.3 83.5 91.3 80.2 82.6 78.9 67.0
下载: 导出CSV
表 6 不同模块的消融研究
Table 6 Ablation studies about different modules
Method RFConv WAP JL mAcc (%) OA (%) 0 × × × 90.2 92.9 1 √ × × 91.0 93.3 2 √ √ × 91.1 93.5 3 √ × √ 91.1 93.4 4 √ √ √ 91.2 93.6
下载: 导出CSV
表 8 不同特征关系的消融实验
Table 8 Ablation studies about different relationship between features
Method ${e_{ij}}$ $di{s_{ij}}$ ${l_{ij}}$ ${s_{ij}}$ OA (%) 1 √ × × × 93.3 2 √ × √ × 93.5 3 √ × √ √ 93.6 4 √ √ × × 93.1 5 × √ √ √ 93.5 6 √ √ √ √ 93.2
下载: 导出CSV
-
[1] Guo Y, Bennamoun M, Sohel F, Min L, Wan J. 3D object recognition in cluttered scenes with local surface features: a survey. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 36(11): 2270-2287 [2] 田永林, 沈宇, 李强, 王飞跃. 平行点云: 虚实互动的点云生成与三维模型进化方法. 自动化学报, 2020, 46(12): 2572-2582Tian Yong-Lin, Shen Yu, Li Qiang, Wang Fei-Yue. Parallel point clouds: point clouds generation and 3d model evolution via virtual-real interaction. Acta Automatica Sinica, 2020, 46(12): 2572-2582 [3] Qi C R, Su H, Mo K, Guibas L J. Pointnet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI: IEEE, 2017. 77−85 [4] Guo Y, Wang H, Hu Q, Liu H, Bennamoun M. Deep learning for 3d point clouds: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. 43(12): 4438-4364 [5] Su H, Maji S, Kalogerakis E, Learned-Miller E. Multi-view convolutional neural networks for 3D shape recognition. In: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015. 945−953 [6] Yang Z, Wang L. Learning relationships for multi-view 3d object recognition. In: Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Seoul, South Korea: IEEE, 2019. 7504-7513 [7] 徐思雨, 祝继华, 田智强, 李垚辰, 庞善民. 逐步求精的多视角点云配准方法. 自动化学报, 2019, 45(8): 1486-1494Xu Si-Yu, Zhu Ji-Hua, Tian Zhi-Qiang, Li Yao-Chen, Pang Shan-Min. Stepwise refinement approach for registration of multi-view point sets. Acta Automatica Sinica, 2019, 45(8): 1486-1494 [8] Maturana D, Scherer S. Voxnet: a 3d convolutional neural network for real-time object recognition. In: Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Hamburg, Germany: IEEE, 2015. 922-928 [9] Riegler G, Osman Ulusoy A, Geiger A. OctNet: Learning deep 3D representations at high resolutions. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI: IEEE, 2017. 6620−6629 [10] Qi C R, Yi L, Su H, Guibas L J. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 2017 Advances in Neural Information Processing Systems. Long Beach, CA, 2017. 5099−5108 [11] Wang C, Samari B, Siddiqi K. Local spectral graph convolution for point set feature learning. In: Proceedings of the 2018 European Conference on Computer Vision (ECCV). Munich, Germany, 2018. 56-71 [12] Wang Y, Sun Y, Liu Z, Sarma S E, Bronstein M M, Solomon J M. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics, 2019, 38(5): 1-12 [13] Liu Y, Fan B, Xiang S, Pan C. Relation-shape convolutional neural network for point cloud analysis. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA: IEEE, 2019.8895-8904 [14] Wang L, Huang Y, Hou Y, Zhang S, Shan J. Graph attention convolution for point cloud semantic segmentation. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA: IEEE, 2019.10288-10297 [15] Chen C, Fragonara L Z, Tsourdos A. Gapointnet: graph attention based point neural network for exploiting local feature of point cloud. Neurocomputing, 2021, 438: 122-132 doi: 10.1016/j.neucom.2021.01.095 [16] Guo M H, Cai J X, Liu Z N, Mu T J, Martin R R, Hu S M. Pct: point cloud transformer. Computational Visual Media, 2021, 7(2): 187-299 doi: 10.1007/s41095-021-0229-5 [17] Wen Y, Zhang K, Li Z, Qiao Y. A discriminative feature learning approach for deep face recognition. In: Proceedings of the 2016 European Conference on Computer Vision. Amsterdam, Netherlands, 2016. 499-515 [18] Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, et al. 3D shapenets: a deep representation for volumetric shapes. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA: IEEE, 2015.1912-1920 [19] Yi L, Kim V G, Ceylan D, Shen I C, Yan M, Su H, et al. A scalable active framework for region annotation in 3d shape collections. ACM Transactions on Graphics, 2016, 35(6): 1-12 [20] Uy M A, Pham Q H, Hua B S, Nguyen T, Yeung S K. Revisiting point cloud classification: a new benchmark dataset and classification model on real-world data. In: Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, South Korea: IEEE, 2019.1588-1597 [21] Simonovsky M, Komodakis N. Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI: IEEE, 2017.3693-3702 [22] 田钰杰, 管有庆, 龚锐. 一种鲁棒的多特征点云分类分割深度神经网络. 计算机工程, 2021, 47(11): 234-240Tian Yu-Jie, Guan You-Qing, Gong Rui. A robust deep neural network for multi-feature point cloud classification and segmentation. Computer Engineering, 2021, 47(11): 234-240 [23] Li Y, Bu R, Sun M, Chen B. Pointcnn: convolution on x-transformed points. In: Proceeding of the 2018 Advances in Neural Information Processing System. Montreal, Canada, 2018. 828-838 [24] Liu X, Han Z, Liu Y, Zwicker M. Point2sequence: learning the shape representation of 3d point clouds with an attention-based sequence to sequence network. In: Proceedings of the 2019 AAAI Conference on Artificial Intelligence. Honolulu, HI: AAAI, 2019.8778-8785 [25] Engel N, Belagiannis V, Dietmayer K. Point transformer. IEEE Access, 2021, 9: 134826-134840 doi: 10.1109/ACCESS.2021.3116304 [26] 许翔, 帅惠, 刘青山. 基于卦限卷积神经网络的3D点云分析. 自动化学报, 2021, 47(12): 2791−2800Xu Xiang, Shuai Hui, Liu Qing-Shan. Octant convolutional neural network for 3D point cloud analysis. Acta Automatica Sinica, 2021, 47(12): 2791−2800 [27] Qiu S, Anwar S, Barnes N. Dense-resolution network for point cloud classification and segmentation. In: Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). Electr Network: IEEE, 2021.3812-3821 [28] Zhou H, Feng Y, Fang M, Wei M, Qin J, Lu T. Adaptive graph convolution for point cloud analysis. In: Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, Canada: IEEE, 2021. 4965-4974 [29] Xu Y, Fan T, Xu M, Long Z, Yu Q. Spidercnn: deep learning on point sets with parameterized convolutional filters. In: Proceedings of the 2018 European Conference on Computer Vision. Munich, Germany, 2018.90-105 [30] Thomas H, Qi C R, Deschaud J E, Marcotegui B, Goulette F, Guibas L. Kpconv: flexible and deformable convolution for point clouds. In: Proceedings of the 2020 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, South Korea: IEEE, 2020.6420-6429 -
点击查看大图
计量
- 文章访问数: 576
- HTML全文浏览量: 212
- 被引次数: 0
