Abstract

Object detection and completion in 3D have shown considerable potential in the field of autonomous vehicles, robots, and mixed reality, but it heavily depends on dense 3D annotations, which is difficult to scale. In this work, the authors have proposed a feedback-driven learning framework for 3D perception with lim-ited supervision. It generates pseudo-labels using weak supervision and refines them using a closed-loop process, which includes the use of vision foundation models along with geometry-aware con-straints. This has shown considerable potential in the field of 3D object detection and completion in a scalable and robust manner.

References

• Y. Yan et al., “SECOND: Sparsely Embedded Convolutional Detection,” Sensors, 2018.
• Y. Liu et al., “PETR: Position Embedding Transformation for Multi-View 3D Object Detection,” in Proceedings of the Eu-ropean Conference on Computer Vision (ECCV), 2022.
• Z. Li et al., “BEVFormer: Learning Bird’s-Eye-View Rep-resentation from Multi-Camera Images via Spatio-Temporal Transformers,” in Proceedings of the European Conference on Computer Vision (ECCV), 2022.
• Y. Liu et al., “PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images,” arXiv preprint arXiv:2301.02604, 2023.
• X. Xu et al., “Back to Reality: Weakly-Supervised 3D Object Detection with Shape-Guided Label Enhancement,” in Pro-ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
• G. Zhang et al., “General Geometry-Aware Weakly Super-vised 3D Object Detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
• M. Li et al., “WI3D: Weakly Incremental 3D Detection via Vision Foundation Models,” IEEE Transactions on Multime-dia, 2025.
• A. Yang et al., “FOMO-3D: Using Vision Foundation Mod-els for Long-Tailed 3D Object Detection,” in Proceedings of the International Conference on Learning Representations (ICLR), 2025.
• L. Wu et al., “Unsupervised 3D Point Cloud Completion via Multi-View Adversarial Learning,” IEEE Transactions on Vi-sualization and Computer Graphics, 2025.
• H. Ma et al., “USSPA: Unsupervised Symmetric Shape-Preserving Autoencoder for 3D Point-Cloud Completion,” in Proceedings of the IEEE/CVF Conference on Computer Vi-sion and Pattern Recognition (CVPR), 2023. .
• S. Chen et al., “SymmCompletion: Symmetry-Guided Point-Cloud Completion,” in Proceedings of the IEEE/CVF Inter-national Conference on Computer Vision (ICCV), 2025.
• B. Wilson et al., “What Matters in Range-View 3D Object Detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
• A. Lang et al., “PointPillars: Fast Encoders for Object Detec-tion from Point Clouds,” in Proceedings of the IEEE Confer-ence on Computer Vision and Pattern Recognition (CVPR), 2019.
• S. Shi et al., “PointRCNN: 3D Object Proposal Genera-tion and Detection from Point Cloud,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni-tion (CVPR), 2019.
• S. Shi et al., “PV-RCNN: Point-Voxel Feature Set Abstrac-tion for 3D Object Detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
• Y. Yin et al., “CenterPoint: Center-Based 3D Object Detec-tion and Tracking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
• C. R. Qi et al., “VoteNet: A Deep Learning Framework for 3D Object Detection,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019.
• P. Philion and S. Fidler, “Lift, Splat, Shoot: Encoding Im-ages from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D,” in Proceedings of the European Conference on Com-puter Vision (ECCV), 2020.
• Y. Huang et al., “BEVDet: High-Performance Multi-Camera 3D Object Detection in Bird’s-Eye View,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2022.
• H. Li et al., “BEVDepth: Acquisition of Reliable Bird’s-Eye-View Representations via Depth Estimation,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
• Z. Li et al., “BEVFusion: Multi-Task Multi-Sensor Fusion with Unified BEV Representation,” in Proceedings of Ad-vances in Neural Information Processing Systems (NeurIPS), 2023.
• Z. Hou et al., “Multi-View 3D Object Detection with Sparse Supervision,” in Proceedings of the IEEE International Con-ference on Computer Vision (ICCV), 2021.
• W. Yuan et al., “PCN: Point Completion Network,” in Pro-ceedings of the International Conference on 3D Vision (3DV), 2018.
• Y. Yang et al., “FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation,” in Proceedings of the IEEE Confer-ence on Computer Vision and Pattern Recognition (CVPR), 2018.
• H. Xie et al., “GRNet: Gridding Residual Network for Dense Point Cloud Completion,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
• X. Yu et al., “PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021.
• X. Yu et al., “AdaPoinTr: Adaptive Geometry-Aware Point Cloud Completion,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
• L. Tchapmi et al., “TopNet: Structural Point Cloud Decoder,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
• Y. Huang et al., “PF-Net: Point Fractal Network for 3D Point Cloud Completion,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
• S. Xiang et al., “SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021.
• Z. Zhou et al., “SeedFormer: Patch Seeds Based Point Cloud Completion with Upsample Transformer,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR), 2023.
• Z. Chen et al., “PointLDM: Latent Diffusion Models for Point Cloud Completion,” IEEE Transactions on Multimedia, 2024.
• A. Milioto et al., “RangeNet++: Fast and Accurate LiDAR Semantic Segmentation,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.
• X. Zhu et al., “Cylindrical and Asymmetrical Convolution for LiDAR Segmentation,” in Proceedings of the IEEE Confer-ence on Computer Vision and Pattern Recognition (CVPR), 2021.
• J. Kim et al., “Range-View Learning for LiDAR-Based 3D Perception,” IEEE Transactions on Intelligent Vehicles, 2023.
• S. Patel et al., “Efficient Range-View Representation for Real-Time LiDAR-Based 3D Detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni-tion (CVPR), 2024.