基于信息融合的智能网联汽车安全交互决策

黄昭彦; 杨烁; 吴建华; 范佳琦; 田炜; 殷翔; 方浩; 褚洪庆; 高炳钊

doi:10.16383/j.aas.c240680

基于信息融合的智能网联汽车安全交互决策

doi: 10.16383/j.aas.c240680 cstr: 32138.14.j.aas.c240680

黄昭彦^1,,
杨烁^2,,
吴建华^1,,
范佳琦^3,,
田炜^1,,
殷翔^4,,
方浩^5,,
褚洪庆^1,,
高炳钊^{1, 3,}

1.
同济大学汽车学院上海 201804 中国
2.
宾夕法尼亚大学电气与系统工程系费城 19104 美国
3.
同济大学上海自主智能无人系统科学中心上海 201210 中国
4.
上海交通大学自动化与感知学院上海 200240 中国
5.
北京理工大学自动化学院北京 100081 中国

基金项目: 国家重点研发计划(2023YFB2504400), 国家自然科学基金(62373289, 62473291), 中央高校基本科研业务费专项资金资助

详细信息

作者简介:
黄昭彦：同济大学汽车学院博士研究生. 主要研究方向为自动驾驶安全决策与规划. E-mail: huangzhaoyan@tongji.edu.cn

杨烁：宾夕法尼亚大学电气与系统工程系博士研究生. 主要研究方向为控制理论, 形式化方法. E-mail: yangs1@seas.upenn.edu

吴建华：同济大学汽车学院硕士研究生. 主要研究方向为端到端的自动驾驶, 视觉−语言−行动模型. E-mail: 2332980@tongji.edu.cn

范佳琦：同济大学上海自主智能无人系统科学中心博士研究生. 主要研究方向为自动驾驶场景理解, 视觉语言模型. E-mail: fanjq@tongji.edu.cn

田炜：同济大学汽车学院副教授. 主要研究方向为自动驾驶感知与预测技术. E-mail: tian-wei@tongji.edu.cn

殷翔：上海交通大学自动化与感知学院教授. 主要研究方向为系统与控制理论, 自主系统和可信人工智能. E-mail: yinxiang@sjtu.edu.cn

方浩：北京理工大学自动化学院教授. 主要研究方向为多智能体协同决策与控制, 智能无人系统的多传感器融合SLAM和可信群体智能中的形式化方法. E-mail: fangh@bit.edu.cn

褚洪庆：同济大学汽车学院副教授. 主要研究方向为网联新能源汽车经济性驾驶策略, 人类驾驶数据引导的汽车安全决策和数据机理混合增强的车辆运动控制. E-mail: chuhongqing@tongji.edu.cn

高炳钊：同济大学汽车学院教授. 主要研究方向为汽车动力传动系统优化, 汽车控制与智能化. 本文通信作者. E-mail: gaobz@tongji.edu.cn

计量
- 文章访问数: 190
- HTML全文浏览量: 95
- 被引次数: 0
出版历程
- 收稿日期: 2024-10-18
- 录用日期: 2025-04-03
- 网络出版日期: 2025-06-19

Safety Interactive Decision-making for Intelligent Connected Vehicles Based on Information Fusion

HUANG Zhao-Yan^1
,,
YANG Shuo^2
,,
WU Jian-Hua^1
,,
FAN Jia-Qi^3
,,
TIAN Wei^1
,,
YIN Xiang^4
,,
FANG Hao^5
,,
CHU Hong-Qing^1
,,
GAO Bing-Zhao^{1, 3
,}

1.
School of Automotive Studies, Tongji University, Shanghai 201804, China
2.
Department of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia 19104, USA
3.
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University, Shanghai 201210, China
4.
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China
5.
School of Automation, Beijing Institute of Technology, Beijing 100081, China

Funds: Supported by National Key Research and Development Program of China (2023YFB2504400), National Natural Science Foundation of China (62373289, 62473291), and Fundamental Research Funds for Central Universities

More Information

Author Bio:
HUANG Zhao-Yan　Ph.D. candidate at the School of Automotive Studies, Tongji University. His research interest covers safety decision-making and planning for autonomous driving

YANG Shuo　Ph.D. candidate in the Department of Electrical and Systems Engineering, University of Pennsylvania. His research interest covers control theory and formal methods

WU Jian-Hua　Master student at the School of Automotive Studies, Tongji University. His research interest covers end-to-end autonomous driving and vision-language-action models

FAN Jia-Qi　Ph.D. candidate at Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University. Her research interest covers scene understanding of autonomous driving and vision-language model

TIAN Wei　Associate professor at the School of Automotive Studies, Tongji University. His research interest covers perception and prediction technologies for autonomous driving

YIN Xiang　Professor at the School of Automation and Intelligent Sensing, Shanghai Jiao Tong University. His research interest covers systems and control theory, autonomous systems, and trustworthy AI

FANG Hao　Professor at the School of Automation, Beijing Institute of Technology. His research interest covers multi-agent cooperative decision-making and control, multi-sensor fusion SLAM for intelligent unmanned systems, and formal methods in trustworthy swarm intelligence

CHU Hong-Qing　Associate professor at the School of Automotive Studies, Tongji University. His research interest covers economic driving strategies for connected new energy vehicles, human driving data-guided vehicle safety decision-making, and data-mechanism hybrid enhanced vehicle motion control

GAO Bing-Zhao　Professor at the School of Automotive Studies, Tongji University. His research interest covers vehicle power transmission optimization, vehicle control and intelligence. Corresponding author of this paper

摘要

摘要: 在开放交通场景, 智能网联汽车仍然存在安全可信性弱、交互属性不足等关键瓶颈问题. 随着人工智能(AI)的发展和深度学习的突破, AI模型在自动驾驶领域取得了显著成果, 可以应用于自动驾驶中的场景理解和推理. 本文对基于信息融合的智能网联汽车安全交互决策研究进行综述, 首先梳理开放场景交通感知和理解方面的研究, 然后探讨具有社会交互属性的决策规划模型, 最后总结针对AI模型幻觉的安全验证技术, 通过结合三方面研究, 充分利用AI模型的强大能力实现“熟练司机”驾驶技能, 并讨论安全保障技术, 弥补AI模型“偶尔犯错”的不足, 有望解决自动驾驶安全长尾问题, 进一步推动自动驾驶技术的发展.
- 智能网联系统 /
- 自动驾驶 /
- 信息融合 /
- 交互决策
Abstract: In open traffic scenarios, intelligent connected vehicles still face critical bottlenecks such as weak safety and reliability, and insufficient interactive attributes. With the advancement of artificial intelligence (AI) and breakthroughs in deep learning, AI models have made significant advancements in the field of autonomous driving, applicable to scene understanding and reasoning in autonomous driving. This paper provides a comprehensive review of the research on safety interactive decision-making for intelligent connected vehicles based on information fusion. It begins by organizing research on traffic perception and understanding in open scenarios, then explores decision-making and planning models with social interaction attributes, and concludes with an examination of safety verification techniques for AI model hallucinations. By integrating research in these three areas, fully leveraging the powerful capabilities of AI models to achieve the driving skills of “skilled human drivers”, and discussing safety assurance technologies to compensate for the “occasional mistakes” of AI models, it is hoped to address the long-tail safety issues in autonomous driving and further advance the development of autonomous driving technology.
- Intelligent connected system /
- autonomous driving /
- information fusion /
- interactive decision-making
注释:

1) 1¹ 数据来源于《盖世汽车研究院Robotaxi产业研究报告(2023版)》

HTML全文

图 1 开放场景交通感知和理解结构^{[21, 29-30]}

Fig. 1 Open scene traffic perception and understanding structure^{[21, 29-30]}

下载: 全尺寸图片幻灯片

图 2 基于强化学习/模仿学习的端到端自动驾驶体系结构

Fig. 2 An end-to-end autonomous driving architecture based on reinforcement learning/imitation lerning

下载: 全尺寸图片幻灯片

图 3 端到端自动驾驶规划框架

Fig. 3 An end-to-end autonomous driving planning architecture

下载: 全尺寸图片幻灯片

图 4 自监督学习范式下具有社会交互属性的决策规划模型

Fig. 4 Decision planning model with social interaction attribute under self-supervised learning paradigm

下载: 全尺寸图片幻灯片

图 5 基于信息融合的智能网联汽车安全交互决策技术路线

Fig. 5 An information fusion-based safety interaction decision-making technical roadmap for intelligent connected vehicles

下载: 全尺寸图片幻灯片

参考文献(141)

[1]	新华社. 目前全国已开放智能网联汽车测试道路里程超过 15000 公里. 人民日报海外版, 2023-06-23 (01) The Xinhua News Agency. The country has opened intelligent connected car test road mileage of more than 15000 kilometers. People's Daily Overseas Edition, 2023-06-23 (01
[2]	Abdel-Aty M, Ding S. A matched case-control analysis of autonomous vs human-driven vehicle accidents. Nature Communications, 2024, 15(1): Article No. 4931 doi: 10.1038/s41467-024-48526-4
[3]	Document for full self-driving capability [Online], available: https://www.tesla.com/support/full-self-drivingsubscriptions, May 8, 2025 Document for full self-driving capability [Online], available: https://www.tesla.com/support/full-self-drivingsubscriptions, May 8, 2025
[4]	2023 disengagement reports (California department of motor vehicles, 2024) [Online], available: https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomousvehicles/disengagement-reports/, May 8, 2025 2023 disengagement reports (California department of motor vehicles, 2024) [Online], available: https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomousvehicles/disengagement-reports/, May 8, 2025
[5]	FSD community tracker [Online], available: https://www.teslafsdtracker.com/home, May 8, 2025 FSD community tracker [Online], available: https://www.teslafsdtracker.com/home, May 8, 2025
[6]	National Highway Traffic Safety Administration. Overview of motor vehicle traffic crashes in 2022 [Online], available: https://trid.trb.org/View/2362011, June 19, 2025
[7]	Kusano K D, Scanlon J M, Chen Y H, McMurry T L, Chen R S, Gode T, et al. Comparison of Waymo rider-only crash data to human benchmarks at 7.1 million miles. Traffic Injury Prevention, 2024, 25(1): S66−S77
[8]	Road vehicles-safety of the intended functionality [Online], available: https://www.iso.org/standard/70939.html, May 8, 2025 Road vehicles-safety of the intended functionality [Online], available: https://www.iso.org/standard/70939.html, May 8, 2025
[9]	Feng S, Sun H W, Yan X T, Zhu H J, Zou Z X, Shen S Y, et al. Dense reinforcement learning for safety validation of autonomous vehicles. Nature, 2023, 615(7953): 620−627 doi: 10.1038/s41586-023-05732-2
[10]	Bozga M, Iosif R, Sifakis J. Verification of component-based systems with recursive architectures. Theoretical Computer Science, 2023, 940: 146−175 doi: 10.1016/j.tcs.2022.10.022
[11]	Wang W S, Wang L T, Zhang C Y, Liu C L, Sun L J. Social interactions for autonomous driving: A review and perspectives. Foundations and Trends^® in Robotics, 2022, 10(3−4): 198−376
[12]	Li D, Huang Y L, Qian L X. Potential adoption of robotaxi service: The roles of perceived benefits to multiple stakeholders and environmental awareness. Transport Policy, 2022, 126: 120−135 doi: 10.1016/j.tranpol.2022.07.004
[13]	The Select Committee on Artificial Intelligence of the National Science and Technology Council. National Artificial Intelligence Research and Development Strategic Plan 2023 Update, National Science and Technology Council, USA, 2023. The Select Committee on Artificial Intelligence of the National Science and Technology Council. National Artificial Intelligence Research and Development Strategic Plan 2023 Update, National Science and Technology Council, USA, 2023.
[14]	Philion J, Fidler S. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3D. In: Proceedings of the 16th European Conference on Computer Vision (ECCV). Glasgow, UK: Springer, 2020. 194−210
[15]	Reading C, Harakeh A, Chae J, Waslander S L. Categorical depth distribution network for monocular 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2021. 8551−8560
[16]	Huang J J, Huang G, Zhu Z, Ye Y, Du D L. BEVDet: High-performance multi-camera 3D object detection in bird-eye-view. arXiv preprint arXiv: 2112.11790, 2021.
[17]	Pan B W, Sun J K, Leung H Y T, Andonian A, Zhou B L. Cross-view semantic segmentation for sensing surroundings. IEEE Robotics and Automation Letters, 2020, 5(3): 4867−4873 doi: 10.1109/LRA.2020.3004325
[18]	Roddick T, Cipolla R. Predicting semantic map representations from images using pyramid occupancy networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 11135−11144
[19]	Gong S, Ye X Q, Tan X, Wang J D, Ding E R, Zhou Y, et al. GitNet: Geometric prior-based transformation for birds-eye-view segmentation. In: Proceedings of the 17th European Conference on Computer Vision (ECCV). Tel Aviv, Israel: Springer, 2022. 396−411
[20]	Wang Y, Guizilini V, Zhang T Y, Wang Y L, Zhao H, Solomon J. DETR3D: 3D object detection from multi-view images via 3D-to-2D queries. In: Proceedings of the 5th Conference on Robot Learning. London, UK: PMLR, 2021. 180−191
[21]	Li Z Q, Wang W H, Li H Y, Xie E Z, Sima C H, Lu T, et al. BEVFormer: Learning bird's-eye-view representation from multi-camera images via spatiotemporal transformers. In: Proceedings of the 17th European Conference on Computer Vision (ECCV). Tel Aviv, Israel: Springer, 2022. 1−18
[22]	Yang C Y, Chen Y T, Tian H, Tao C X, Zhu X Z, Zhang Z X, et al. BEVFormer v2: Adapting modern image backbones to bird's-eye-view recognition via perspective supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada: IEEE, 2023. 17830−17839
[23]	Liu Y F, Wang T C, Zhang X Y, Sun J. PETR: Position embedding transformation for multi-view 3D object detection. In: Proceedings of the 17th European Conference on Computer Vision (ECCV). Tel Aviv, Israel: Springer, 2022. 531−548
[24]	Liu Y F, Yan J J, Jia F, Li S L, Gao A Q, Wang T C, et al. PETRv2: A unified framework for 3D perception from multi-camera images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Paris, France: IEEE, 2023. 3239−3249
[25]	Lang A H, Vora S, Caesar H, Zhou L B, Yang J, Beijbom O. BEVFormer v2: Adapting modern image backbones to bird＇s-eye-view recognition via perspective supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). California, USA: IEEE, 2019. 12697−12705 Lang A H, Vora S, Caesar H, Zhou L B, Yang J, Beijbom O. BEVFormer v2: Adapting modern image backbones to bird's-eye-view recognition via perspective supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). California, USA: IEEE, 2019. 12697−12705
[26]	Chen X Z, Ma H M, Wan J, Li B, Xia T. Multi-view 3D object detection network for autonomous driving. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 65267−6534
[27]	Zhou Y, Tuzel O. VoxelNet: End-to-end learning for point cloud based 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, USA: IEEE, 2018. 4490−4499
[28]	秦超, 王亚飞, 张宇超, 殷承良. 基于极端稀疏激光点云和RGB图像的3D目标检测. 激光与光电子学进展, 2022, 59(18): Article No. 1828004 Qin Chao, Wang Ya-Fei, Zhang Yu-Chao, Yin Cheng-Liang. 3D object detection based on extremely sparse laser point cloud and RGB images. Laser & Optoelectronics Progress, 2022, 59(18): Article No. 1828004
[29]	Liu Z J, Tang H T, Amini A, Yang X Y, Mao H Z, Rus D L, et al. BEVFusion: Multi-task multi-sensor fusion with unified bird's-eye view representation. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). London, UK: IEEE, 2023. 2774−2781
[30]	Huang J J, Huang G. BEVDet4D: Exploit temporal cues in multi-camera 3D object detection. arXiv preprint arXiv: 2203.17054, 2022.
[31]	Li Y W, Chen Y L, Qi X J, Li Z M, Sun J, Jia J Y. Unifying voxel-based representation with transformer for 3D object detection. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2022. Article No. 1340
[32]	Pang S, Morris D, Radha H. CLOCs: Camera-LiDAR object candidates fusion for 3D object detection. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Las Vegas, USA: IEEE, 2020. 10386−10393
[33]	Li Q, Wang Y, Wang Y L, Zhao H. HDMapNet: An online HD map construction and evaluation framework. In: Proceedings of the International Conference on Robotics and Automation (ICRA). Philadelphia, USA: IEEE, 2022. 4628−4634
[34]	吴绍斌, 耿家琳, 吴超, 闫泽新, 陈恺宇. 基于多帧信息的多传感器融合三维目标检测. 北京理工大学学报, 2023, 43(12): 1282−1289 Wu Shao-Bin, Geng Jia-Lin, Wu Chao, Yan Ze-Xin, Chen Kai-Yu. Multi-sensor fusion 3D object detection based on multi-frame information. Transactions of Beijing Institute of Technology, 2023, 43(12): 1282−1289
[35]	Qin Z Q, Chen J Y, Chen C, Chen X Z, Li X. UniFusion: Unified multi-view fusion Transformer for spatial-temporal representation in bird's-eye-view. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Paris, France: IEEE, 2023. 8656−8665
[36]	Sun W C, Lin X W, Shi Y N, Zhang C, Wu H R, Zheng S F. SparseDrive: End-to-end autonomous driving via sparse scene representation. arXiv preprint arXiv: 2405.19620, 2024.
[37]	Weng X S, Ivanovic B, Wang Y, Wang Y, Pavone M. PARA-drive: Parallelized architecture for real-time autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2024. 15449−15458
[38]	Li P D, Cui D X. Does end-to-end autonomous driving really need perception tasks? arXiv preprint arXiv: 2409.18341, 2024. Li P D, Cui D X. Does end-to-end autonomous driving really need perception tasks? arXiv preprint arXiv: 2409.18341, 2024.
[39]	Kuefler A, Morton J, Wheeler T, Kochenderfer M. Imitating driver behavior with generative adversarial networks. In: Proceedings of the IEEE Intelligent Vehicles Symposium (IV). Los Angeles, USA: IEEE, 2017. 204−211
[40]	Lu C, Wang H J, Lv C, Gong J W, Xi J Q, Cao D P. Learning driver-specific behavior for overtaking: A combined learning framework. IEEE Transactions on Vehicular Technology, 2018, 67(8): 6788−6802 doi: 10.1109/TVT.2018.2820002
[41]	Acerbo F S, Swevers J, Tuytelaars T, Son T D. Evaluation of MPC-based imitation learning for human-like autonomous driving. IFAC-PapersOnLine, 2023, 56(2): 4871−4876 doi: 10.1016/j.ifacol.2023.10.1257
[42]	Ahmedov H B, Yi D W, Sui J. Application of a brain-inspired deep imitation learning algorithm in autonomous driving. Software Impacts, 2021, 10: Article No. 100165 doi: 10.1016/j.simpa.2021.100165
[43]	徐优志. 自动驾驶车辆高速道路环境下超车行为决策研究 [硕士学位论文], 北京理工大学, 中国, 2016. Xu You-Zhi. Decision-making modeling of overtaking behavior for autonomous vehicles on freeway environment [Master thesis], Beijing Institute of Technology, China, 2016.
[44]	陈鸿军. 基于模仿学习的智能车辆行为决策与运动控制方法研究 [硕士学位论文], 国防科技大学, 中国, 2019. Chen Hong-Jun. Research on behavior decision-making and motion control methods based on imitation learning for intelligent vehicles [Master thesis], National University of Defense Technology, China, 2019.
[45]	Xu H Z, Gao Y, Yu F, Darrell T. End-to-end learning of driving models from large-scale video datasets. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 3530−3538
[46]	Bhattacharyya R, Wulfe B, Phillips D J, Kuefler A, Morton J, Senanayake R, et al. Modeling human driving behavior through generative adversarial imitation learning. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(3): 2874−2887 doi: 10.1109/TITS.2022.3227738
[47]	Li Y Z, Song J M, Ermon S. InfoGAIL: Interpretable imitation learning from visual demonstrations. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: Curran Associates Inc., 2017. 3815−3825
[48]	Wang H J, Gao H B, Yuan S H, Zhao H F, Wang K L, Wang X L, et al. Interpretable decision-making for autonomous vehicles at highway on-ramps with latent space reinforcement learning. IEEE Transactions on Vehicular Technology, 2021, 70(9): 8707−8719 doi: 10.1109/TVT.2021.3098321
[49]	Codevilla F, Miiller M, López A, Koltun V, Dosovitskiy A. End-to-end driving via conditional imitation learning. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). Brisbane, Australia: IEEE, 2018. 1−9
[50]	Liang X D, Wang T R, Yang L N, Xing E. CIRL: Controllable imitative reinforcement learning for vision-based self-driving. In: Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018. 604−620
[51]	Chen C Y, Seff A, Kornhauser A, Xiao J X. DeepDriving: Learning affordance for direct perception in autonomous driving. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015. 2722−2730
[52]	Kong J, Pfeiffer M, Schildbach G, Borrelli F. Kinematic and dynamic vehicle models for autonomous driving control design. In: Proceedings of the IEEE Intelligent Vehicles Symposium (IV). Seoul, South Korea: IEEE, 2015. 1094−1099
[53]	Casas S, Sadat A, Urtasun R. MP3: A unified model to map, perceive, predict and plan. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2021. 14398−14407
[54]	Chitta K, Prakash A, Jaeger B, Yu Z H, Renz K, Geiger A. TransFuser: Imitation with Transformer-based sensor fusion for autonomous driving. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(11): 12878−12895 doi: 10.1109/TPAMI.2022.3200245
[55]	Hu Y H, Yang J Z, Chen L, Li K Y, Sima C H, Zhu X Z, et al. Planning-oriented autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada: IEEE, 2023. 17853−17862
[56]	Toromanoff M, Wirbel E, Moutarde F. End-to-end model-free reinforcement learning for urban driving using implicit affordances. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 7151−7160
[57]	Wen L, Duan J L, Li S E, Xu S B, Peng H. Safe reinforcement learning for autonomous vehicles through parallel constrained policy optimization. In: Proceedings of the IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC). Rhodes, Greece: IEEE, 2020. 1−7
[58]	Chitta K, Prakash A, Geiger A. NEAT: Neural attention fields for end-to-end autonomous driving. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, Canada: IEEE, 2021. 15773−15783
[59]	Wu P H, Jia X S, Chen L, Yan J C, Li H Y, Qiao Y. Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2022. Article No. 443
[60]	Prakash A, Chitta K, Geiger A. Multi-modal fusion Transformer for end-to-end autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2021. 7073−7083
[61]	Jia X S, Wu P H, Chen L, Xie J W, He C H, Yan J C, et al. Think Twice before driving: Towards scalable decoders for end-to-end autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada: IEEE, 2023. 21983−21994
[62]	Chen L, Wu P H, Chitta K, Jaeger B, Geiger A, Li H Y. End-to-end autonomous driving: Challenges and frontiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(12): 10164−10183 doi: 10.1109/TPAMI.2024.3435937
[63]	Ngiam J, Caine B, Vasudevan V, Zhang Z D, Chiang H T L, Ling J, et al. Scene Transformer: A unified architecture for predicting multiple agent trajectories. arXiv preprint arXiv: 2106.08417, 2021.
[64]	Renz K, Chitta K, Mercea O B, Koepke A S, Akata Z, Geiger A. PlanT: Explainable planning Transformers via object-level representations. In: Proceedings of the 6th Annual Conference on Robot Learning (CoRL). Auckland, New Zealand: PMLR, 2023. 459−470
[65]	Zhang K P, Feng X L, Wu L, He Z B. Trajectory prediction for autonomous driving using spatial-temporal graph attention transformer. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(11): 22343−22353 doi: 10.1109/TITS.2022.3164450
[66]	Ye T J, Jing W, Hu C Y, Huang S K, Gao L P, Wang J K, et al. FusionAD: Multi-modality fusion for prediction and planning tasks of autonomous driving. arXiv preprint arXiv: 2308.01006, 2023. Ye T J, Jing W, Hu C Y, Huang S K, Gao L P, Wang J K, et al. FusionAD: Multi-modality fusion for prediction and planning tasks of autonomous driving. arXiv preprint arXiv: 2308.01006, 2023.
[67]	Jin B, Liu X Y, Zheng Y P, Li P F, Zhao H, Zhang T, et al. ADAPT: Action-aware driving caption Transformer. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). London, HK: IEEE, 2023. 7554−7561
[68]	Liu H X, Feng S. Curse of rarity for autonomous vehicles. Nature Communications, 2024, 15(1): Article No. 4808 doi: 10.1038/s41467-024-49194-0
[69]	Yan X, Zhang H M, Cai Y J, Guo J M, Qiu W C, Gao B, et al. Forging vision foundation models for autonomous driving: Challenges, methodologies, and opportunities. arXiv preprint arXiv: 2401.08045, 2024.
[70]	Gao H X, Wang Z R, Li Y Q, Long K W, Yang M, Shen Y Q. A survey for foundation models in autonomous driving. arXiv preprint arXiv: 2402.01105, 2024.
[71]	Tian X Y, Gu J R, Li B L, Liu Y C, Wang Y, Zhao Z Y, et al. DriveVLM: The convergence of autonomous driving and large vision-language models. arXiv preprint arXiv: 2402.12289, 2024.
[72]	Bai Z C, Wang P C, Xiao T J, He T, Han Z B, Zhang Z, et al. Hallucination of multimodal large language models: A survey. arXiv preprint arXiv: 2404.18930, 2024.
[73]	Chen T, Kornblith S, Norouzi M, Hinton G. A simple framework for contrastive learning of visual representations. In: Proceedings of the 37th International Conference on Machine Learning. Glasgow, UK: JMLR.org, 2020. Article No. 149
[74]	Misra I, van der Maaten L. Self-supervised learning of pretext-invariant representations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 6706−6716
[75]	Luo C X, Yang X D, Yuille A. Self-supervised pillar motion learning for autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2021. 3182−3191
[76]	Karlsson R, Carballo A, Fujii K, Ohtani K, Takeda K. Predictive world models from real-world partial observations. In: Proceedings of the IEEE International Conference on Mobility, Operations, Services and Technologies (MOST). Detroit, USA: IEEE, 2023. 152−166
[77]	Kingma D P, Welling M. Auto-encoding variational bayes. arXiv preprint arXiv: 1312.6114, 2013. Kingma D P, Welling M. Auto-encoding variational bayes. arXiv preprint arXiv: 1312.6114, 2013.
[78]	Liao Y Y, Xie J, Geiger A. KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2D and 3D. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(3): 3292−3310 doi: 10.1109/TPAMI.2022.3179507
[79]	Hu A, Russell L, Yeo H, Murez Z, Fedoseev G, Kendall A, et al. GAIA-1: A generative world model for autonomous driving. arXiv preprint arXiv: 2309.17080, 2023.
[80]	Wang X F, Zhu Z, Huang G, Chen X Z, Zhu J G, Lu J W. DriveDreamer: Towards real-world-drive world models for autonomous driving. In: Proceedings of the 18th European Conference on Computer Vision. Milan, Italy: Springer, 2023. 55−72
[81]	Zhang L J, Xiong Y W, Yang Z, Casas S, Hu R, Urtasun R. Copilot4D: Learning unsupervised world models for autonomous driving via discrete diffusion. arXiv preprint arXiv: 2311.01017, 2023. Zhang L J, Xiong Y W, Yang Z, Casas S, Hu R, Urtasun R. Copilot4D: Learning unsupervised world models for autonomous driving via discrete diffusion. arXiv preprint arXiv: 2311.01017, 2023.
[82]	Chang H W, Zhang H, Jiang L, Liu C, Freeman W T. MaskGIT: Masked generative image Transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE, 2022. 11305−11315
[83]	van den Oord A, Vinyals O, Kavukcuoglu K. Neural discrete representation learning. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: Curran Associates Inc., 2017. 6309−6318
[84]	Gu T P, Chen G Y, Li J L, Lin C Z, Rao Y M, Zhou J, et al. Stochastic trajectory prediction via motion indeterminacy diffusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE, 2022. 17092−17101
[85]	Dabral R, Mughal M H, Golyanik V, Theobalt C. MoFusion: A framework for denoising-diffusion-based motion synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada: IEEE, 2023. 9760−9770
[86]	Li Z Y, Liang H W, Wang H Q, Zheng X K, Wang J, Zhou P F. A multi-modal vehicle trajectory prediction framework via conditional diffusion model: A coarse-to-fine approach. Knowledge-based Systems, 2023, 280: Article No. 110990 doi: 10.1016/j.knosys.2023.110990
[87]	Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B. High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE, 2022. 10674−10685
[88]	Chen K, Xie E Z, Chen Z, Wang Y B, Hong L Q, Li Z G, et al. GeoDiffusion: Text-prompted geometric control for object detection data generation. arXiv preprint arXiv: 2306.04607, 2023.
[89]	Zheng W Z, Song R Q, Guo X D, Zhang C M, Chen L. GenAD: Generative end-to-end autonomous driving. In: Proceedings of the 18th European Conference on Computer Vision. Milan, Italy: Springer, 2024. 87−104
[90]	Belta C, Yordanov B, Gol E A. Formal Methods for Discrete-time Dynamical Systems. Cham: Springer, 2017.
[91]	Pek C, Manzinger S, Koschi M, Althoff M. Using online verification to prevent autonomous vehicles from causing accidents. Nature Machine Intelligence, 2020, 2(9): 518−528 doi: 10.1038/s42256-020-0225-y
[92]	Yin X, Gao B Z, Yu X. Formal synthesis of controllers for safety-critical autonomous systems: Developments and challenges. Annual Reviews in Control, 2024, 57: Article No. 100940 doi: 10.1016/j.arcontrol.2024.100940
[93]	Donzé A, Ferrère T, Maler O. Efficient robust monitoring for STL. In: Proceedings of the 25th International Conference on Computer Aided Verification. Saint Petersburg, Russia: Springer, 2013. 264−279
[94]	Donzé A, Maler O. Robust satisfaction of temporal logic over real-valued signals. In: Proceedings of the 8th International Conference on Formal Modeling and Analysis of Timed Systems. Klosterneuburg, Austria: Springer, 2010. 92−106
[95]	Fainekos G E, Pappas G J. Robustness of temporal logic specifications for continuous-time signals. Theoretical Computer Science, 2009, 410(42): 4262−4291 doi: 10.1016/j.tcs.2009.06.021
[96]	Deshmukh J V, Donzé A, Ghosh S, Jin X Q, Juniwal G, Seshia S A. Robust online monitoring of signal temporal logic. Formal Methods in System Design, 2017, 51(1): 5−30 doi: 10.1007/s10703-017-0286-7
[97]	Zhang Z Y, Arcaini P, Xie X. Online reset for signal temporal logic monitoring. IEEE Transactions on Computer-aided Design of Integrated Circuits and Systems, 2022, 41(11): 4421−4432 doi: 10.1109/TCAD.2022.3197693
[98]	Yu W H, Zhao C X, Wang H, Liu J X, Ma X H, Yang Y K, et al. Online legal driving behavior monitoring for self-driving vehicles. Nature Communications, 2024, 15(1): Article No. 408 doi: 10.1038/s41467-024-44694-5
[99]	Sahin Y E, Quirynen R, di Cairano S. Autonomous vehicle decision-making and monitoring based on signal temporal logic and mixed-integer programming. In: Proceedings of the American Control Conference (ACC). Denver, USA: IEEE, 2020. 454−459
[100]	Arechiga N. Specifying safety of autonomous vehicles in signal temporal logic. In: Proceedings of the IEEE Intelligent Vehicles Symposium (IV). Paris, France: IEEE, 2019. 58−63
[101]	Hekmatnejad M, Yaghoubi S, Dokhanchi A, Amor H B, Shrivastava A, Karam L, et al. Encoding and monitoring responsibility sensitive safety rules for automated vehicles in signal temporal logic. In: Proceedings of the 17th ACM-IEEE International Conference on Formal Methods and Models for System Design. La Jolla, USA: ACM, 2019. Article No. 6
[102]	Qin X, Deshmukh J V. Clairvoyant monitoring for signal temporal logic. In: Proceedings of the 18th International Conference on Formal Modeling and Analysis of Timed Systems. Vienna, Austria: Springer, 2020. 178−195
[103]	Yu X Y, Dong W J, Li S Y, Yin X. Model predictive monitoring of dynamical systems for signal temporal logic specifications. Automatica, 2024, 160: Article No. 111445 doi: 10.1016/j.automatica.2023.111445
[104]	Ghorbel B, Prabhu V S. Quantitative robustness for signal temporal logic with time-freeze quantifiers. IEEE Transactions on Computer-aided Design of Integrated Circuits and Systems, 2023, 42(12): 4436−4449 doi: 10.1109/TCAD.2023.3283296
[105]	Zhong B Z, Jordan C, Provost J. Extending signal temporal logic with quantitative semantics by intervals for robust monitoring of cyber-physical systems. ACM Transactions on Cyber-physical Systems, 2021, 5(2): Article No. 16
[106]	Finkbeiner B, Fränzle M, Kohn F, Kröger P. A truly robust signal temporal logic: Monitoring safety properties of interacting cyber-physical systems under uncertain observation. Algorithms, 2022, 15(4): Article No. 126 doi: 10.3390/a15040126
[107]	Salamati A, Soudjani S, Zamani M. Data-driven verification of stochastic linear systems with signal temporal logic constraints. Automatica, 2021, 131: Article No. 109781 doi: 10.1016/j.automatica.2021.109781
[108]	Yu X Y, Dong W J, Yin X, Li S Y. Online monitoring of dynamic systems for signal temporal logic specifications with model information. In: Proceedings of the IEEE 61st Conference on Decision and Control (CDC). Cancun, Mexico: IEEE, 2022. 1553−1559
[109]	Yang S, Pappas G J, Mangharam R, Lindemann L. Safe perception-based control under stochastic sensor uncertainty using conformal prediction. In: Proceedings of the 62nd IEEE Conference on Decision and Control (CDC). Singapore: IEEE, 2023. 6072−6078
[110]	Lindemann L, Cleaveland M, Shim G, Pappas G J. Safe planning in dynamic environments using conformal prediction. IEEE Robotics and Automation Letters, 2023, 8(8): 5116−5123 doi: 10.1109/LRA.2023.3292071
[111]	Lekeufack J, Angelopoulos A N, Bajcsy A, Jordan M I, Malik J. Conformal decision theory: Safe autonomous decisions from imperfect predictions. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). Yokohama, Japan: IEEE, 2024. 11668−11675
[112]	Cleaveland M, Lee I, Pappas G J, Lindemann L. Conformal prediction regions for time series using linear complementarity programming. In: Proceedings of the 38th AAAI Conference on Artificial Intelligence. Vancouver, Canada: AAAI Press, 2023. Article No. 2341
[113]	Dixit A, Lindemann L, Wei S X, Cleaveland M, Pappas G J, Burdick J W. Adaptive conformal prediction for motion planning among dynamic agents. In: Proceedings of the 5th Annual Learning for Dynamics and Control Conference. Philadelphia, USA: PMLR, 2023. 300−314 Dixit A, Lindemann L, Wei S X, Cleaveland M, Pappas G J, Burdick J W. Adaptive conformal prediction for motion planning among dynamic agents. In: Proceedings of the 5th Annual Learning for Dynamics and Control Conference. Philadelphia, USA: PMLR, 2023. 300−314
[114]	Yu X Y, Zhao Y Q, Yin X, Lindemann L. Signal temporal logic control synthesis among uncontrollable dynamic agents with conformal prediction. arXiv preprint arXiv: 2312.04242, 2023.
[115]	Lindemann L, Qin X, Deshmukh J V, Pappas G J. Conformal prediction for STL runtime verification. In: Proceedings of the ACM/IEEE 14th International Conference on Cyber-physical Systems (with CPS-IoT Week 2023). New York, USA: ACM, 2023. 142−153
[116]	Sinha R, Schmerling E, Pavone M. Closing the loop on runtime monitors with fallback-safe MPC. In: Proceedings of the 62nd IEEE Conference on Decision and Control (CDC). Singapore: IEEE, 2023. 6533−6540
[117]	Yoo C, Belta C. Control with probabilistic signal temporal logic. arXiv preprint arXiv: 1510.08474, 2015.
[118]	陈杰, 吕梓亮, 黄鑫源, 洪奕光. 非线性系统的安全分析与控制: 障碍函数方法. 自动化学报, 2023, 49(3): 567−579 Chen Jie, Lv Zi-Liang, Huang Xin-Yuan, Hong Yi-Guang. Safety analysis and safety-critical control of nonlinear systems: Barrier function approach. Acta Automatica Sinica, 2023, 49(3): 567−579
[119]	Ames A D, Xu X R, Grizzle J W, Tabuada P. Control barrier function based quadratic programs for safety critical systems. IEEE Transactions on Automatic Control, 2017, 62(8): 3861−3876 doi: 10.1109/TAC.2016.2638961
[120]	Xu X R, Grizzle J W, Tabuada P, Ames A D. Correctness guarantees for the composition of lane keeping and adaptive cruise control. IEEE Transactions on Automation Science and Engineering, 2018, 15(3): 1216−1229 doi: 10.1109/TASE.2017.2760863
[121]	Xiao W, Belta C. High-order control barrier functions. IEEE Transactions on Automatic Control, 2022, 67(7): 3655−3662 doi: 10.1109/TAC.2021.3105491
[122]	Lyu Y W, Luo W H, Dolan J M. Probabilistic safety-assured adaptive merging control for autonomous vehicles. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). Xi'an, China: IEEE, 2021. 10764−10770
[123]	Lyu Y W, Luo W H, Dolan J M. Adaptive safe merging control for heterogeneous autonomous vehicles using parametric control barrier functions. In: Proceedings of the IEEE Intelligent Vehicles Symposium (IV). Aachen, Germany: IEEE, 2022. 542−547
[124]	He S Y, Zeng J, Sreenath K. Autonomous racing with multiple vehicles using a parallelized optimization with safety guarantee using control barrier functions. In: Proceedings of the International Conference on Robotics and Automation (ICRA). Philadelphia, USA: IEEE, 2022. 3444−3451
[125]	Rosolia U, Borrelli F. Learning how to autonomously race a car: A predictive control approach. IEEE Transactions on Control Systems Technology, 2020, 28(6): 2713−2719 doi: 10.1109/TCST.2019.2948135
[126]	Alshiekh M, Bloem R, Ehlers R, Könighofer B, Niekum S, Topcu U. Safe reinforcement learning via shielding. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. New Orleans, USA: AAAI Press, 2018. Article No. 326
[127]	Jansen N, Könighofer B, Junges S, Serban A C, Bloem R. Safe reinforcement learning via probabilistic shields. arXiv preprint arXiv: 1807.06096, 2018.
[128]	Hunt N, Fulton N, Magliacane S, Hoang T N, Das S, Armando S L. Verifiably safe exploration for end-to-end reinforcement learning. In: Proceedings of the 24th International Conference on Hybrid Systems: Computation and Control. Nashville, USA: ACM, 2021. Article No. 14
[129]	Yin J, Dawson C, Fan C C, Tsiotras P. Shield model predictive path integral: A computationally efficient robust MPC method using control barrier functions. IEEE Robotics and Automation Letters, 2023, 8(11): 7106−7113 doi: 10.1109/LRA.2023.3315211
[130]	Kochdumper N, Krasowski H, Wang X, Bak S, Althoff M. Provably safe reinforcement learning via action projection using reachability analysis and polynomial zonotopes. IEEE Open Journal of Control Systems, 2023, 2: 79−92 doi: 10.1109/OJCSYS.2023.3256305
[131]	Hsu K C, Ren A Z, Nguyen D P, Majumdar A, Fisac J F. Sim-to-Lab-to-Real: Safe reinforcement learning with shielding and generalization guarantees. Artificial Intelligence, 2023, 314: Article No. 103811 doi: 10.1016/j.artint.2022.103811
[132]	Wolff E M, Murray R M. Optimal control of nonlinear systems with temporal logic specifications. Robotics Research: The 16th International Symposium ISRR. Cham: Springer, 2016. 21−37
[133]	Aasi E, Vasile C I, Belta C. A control architecture for provably-correct autonomous driving. In: Proceedings of the American Control Conference (ACC). New Orleans, USA: IEEE, 2021. 2913−2918
[134]	Charitidou M, Dimarogonas D V. Receding horizon control with online barrier function design under signal temporal logic specifications. IEEE Transactions on Automatic Control, 2023, 68(6): 3545−3556 doi: 10.1109/TAC.2022.3195470
[135]	Meng Y, Fan C C. Signal temporal logic neural predictive control. IEEE Robotics and Automation Letters, 2023, 8(11): 7719−7726 doi: 10.1109/LRA.2023.3315536
[136]	Shi W T, Luo X, Hong J L, Zhao C L, Gao B Z, Chen H. Accelerating model predictive control with neural network optimizer. In: Proceedings of the 7th CAA International Conference on Vehicular Control and Intelligence (CVCI). Changsha, China: IEEE, 2023. 1−7
[137]	陈仲瑶, 方浩. 基于线性时序逻辑的智能体不确定行为规划. 中国科学: 技术科学, 2020, 50(5): 516−525 doi: 10.1360/SST-2019-0292 Chen Zhong-Yao, Fang Hao. Probabilistic action planning based on linear temporal logic. Scientia Sinica Technologica, 2020, 50(5): 516−525 doi: 10.1360/SST-2019-0292
[138]	Song Y L, Romero A, Müller M, Koltun V, Scaramuzza D. Reaching the limit in autonomous racing: Optimal control versus reinforcement learning. Science Robotics, 2023, 8(82): Article No. eadg1462 doi: 10.1126/scirobotics.adg1462
[139]	田戴荧, 方浩, 杨庆凯. 信号时序逻辑约束下基于终点回溯的高效规划. 无人系统技术, 2021, 4(1): 44−50 Tian Dai-Ying, Fang Hao, Yang Qing-Kai. Efficient planning based on destination backtracking under signal temporal logic constraints. Unmanned Systems Technology, 2021, 4(1): 44−50
[140]	殷翔, 任晓华, 李少远. 基于强化学习的机器人复杂时序逻辑任务路径规划方法, 中国专利 CN114355947A, April 2022 Yin Xiang, Ren Xiao-Hua, Li Shao-Yuan. Robot complex sequential logic task path planning method based on reinforcement learning, China Patent CN114355947A, April 2022
[141]	Lee K M B, Yoo C, Fitch R. Signal temporal logic synthesis as probabilistic inference. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). Xi'an, China: IEEE, 2021. 5483−5489