Time-Varying Formation Tracking Control for Heterogeneous Air-Sea Unmanned Swarms: A Single Network Adaptive Critic Reinforcement Learning Approach
-
摘要: 针对模型完全未知的异构空海无人集群时变编队跟踪控制问题, 提出基于单网络自适应评判强化学习的分层分布式控制方案. 首先, 为无人装备设计完全分布式自适应观测器, 仅依赖在线交互信息, 即可协同估计期望轨迹. 在此基础上, 引入复合学习技术, 以在有限时间内估计系统不确定性. 其次, 构建分层控制框架, 将异构集群编队控制问题解耦为位置与姿态的上下层控制子问题, 并利用单网络自适应评判强化学习技术为各层设计完全无需模型知识的最优控制策略. 理论分析表明, 提出的控制方案能保证跟踪误差和参数估计误差的一致最终有界. 最后, 仿真实验验证了所提方案在实现异构集群时变编队跟踪方面的有效性.Abstract: The time-varying formation tracking control problem for heterogeneous air-sea unmanned swarms with completely unknown dynamics is considered, a hierarchical distributed control scheme is proposed based on single network adaptive critic (SNAC) reinforcement learning. First, fully distributed adaptive observers are designed for vehicles, enabling them to cooperatively estimate the reference trajectories by utilizing only online exchanged information. Based on this, composite learning is introduced for estimation of system uncertainties within finite time. Then, a hierarchical control framework is established, which decouples the formation control problem for the heterogeneous swarms into upper-layer and lower-layer subproblems of position and attitude, and the SNAC reinforcement learning technique is employed to design model-free optimal control policies for each layer. Theoretical analysis demonstrates that the proposed control scheme guarantees the uniform ultimate boundedness of both the tracking errors and parameter estimation errors. Finally, simulation results verify the effectiveness of the proposed scheme in achieving time-varying formation tracking for the heterogeneous swarms.
-
图 2 参数估计误差范数的演化轨迹
Fig. 2 Evolution Trajectories of the parameter estimation error norms
图 4 跟踪误差和相应的参数估计演化曲线
Fig. 4 Tracking error and corresponding parameter estimation evolution curves
-
[1] 王耀南, 华和安, 张辉, 钟杭, 樊叶心, 梁鸿涛, 常浩, 方勇纯. 性能函数引导的无人机集群深度强化学习控制方法. 自动化学报, 2025, 51(5): 905−916Wang Y, Hua H, Zhang H, Zhong H, Fan Y, Liang H, Chang H, and Fang Y. Performance function-guided deep reinforcement learning control for UAV swarm. Acta Automatica Sinica, 2025, 51(5): 905−916 [2] 华长春, 穆殿瑞, 陈健楠, 魏饶. 无人机引导下多消防机器人协同作业. 自动化学报, 2026, doi: 10.16383/j.aas.c250420.Hua C, Mu D, Chen J, and Wei R. Collaborative operation of multiple firefightingro-bots guided by UAVs. Acta Automatica Sinica, 2026, doi: 10.16383/j.aas.c250420. [3] Wen G. Model identification and control of autonomous surface vessels: An overview of recent advances and future perspectives. IEEE Systems, Man, and Cybernetics Magazine, 2026, 12(1): 65−79 doi: 10.1109/MSMC.2025.3542254 [4] Wen G, Fu J, Wang B, Wang X, and Shen H. Robust formation tracking of ASVs with nonconservative collision avoidance: A control barrier function-based approach. IEEE Transactions on Industrial Electronics, 2025, doi: 10.1109/TIE.2025.3649849. [5] 温广辉, 余星火, 黄廷文, 周艳. 模型参数不确定下多无人艇系统固定时间二分编队跟踪控制. 自动化学报, 2025, 51(3): 669−677Wen G, Yu X, Huang T, and Zhou Y. Fixed-time bipartite formation tracking control for multi-USV systems with uncertain model parameters. Acta Automatica Sinica, 2025, 51(3): 669−677 [6] 江碧涛, 温广辉, 周佳玲, 郑德智. 智能无人集群系统跨域协同技术研究现状与展望. 中国工程科学, 2024, 26(1): 117−126 doi: 10.15302/J-SSCAE-2024.01.015Jiang B, Wen G, Zhou J, and Zheng D. Cross-domain cooperative technology of intelligent unmanned swarm systems: Current status and prospects. Strategic Study of CAE, 2024, 26(1): 117−126 doi: 10.15302/J-SSCAE-2024.01.015 [7] 闫敬, 关新平. 海上无人系统跨域集群发展现状及其关键技术. 自动化学报, 2025, 51(4): 744−761Yan J and Guan X. Development status and key techniques for cross-domain swarm of maritime unmanned systems. Acta Automatica Sinica, 2025, 51(4): 744−761 [8] Wang H, Wang H, and Guo X P. Distributed adaptive fault-tolerant formation control for heterogeneous USV-AUV swarms based on dynamic event triggering. Journal of Marine Science and Engineering, 2025, 13(11): 2116 doi: 10.3390/jmse13112116 [9] Liu H, Huang H, Tian X, and Zhang J. Distributed fixed-time formation control for UAV-USV multiagent systems based on the FEWNN with prescribed performance. Ocean Engineering, 2025, 328: 120996 doi: 10.1016/j.oceaneng.2025.120996 [10] Huang D, Li H, and Li X. Formation of generic UAVs-USVs system under distributed model predictive control scheme. IEEE Transactions on Circuits and Systems II: Express Briefs, 2020, 67(12): 3123−3127 [11] Zhu Y, Li S, Guo G, Yuan P, and Bai J. Formation control of UAV–USV based on distributed event-triggered adaptive MPC with virtual trajectory restriction. Ocean Engineering, 2024, 294: 116850 doi: 10.1016/j.oceaneng.2024.116850 [12] 温广辉, 杨涛, 周佳玲, 付俊杰, 徐磊. 强化学习与自适应动态规划: 从基础理论到多智能体系统中的应用进展综述. 控制与决策, 2023, 38(5): 1200−1230Wen G, Yang T, Zhou J, and Fu J. Reinforcement learning and adaptive/approximate dynamic programming: A survey from theory to applications in multi-agent systems. Control and Decision, 2023, 38(5): 1200−1230 [13] Padhi R, Unnikrishnan N, Wang X, and Balakrishnan S N. A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems. Neural Networks, 2006, 19(10): 1648−1660 [14] Zhou Y, Wen G, Zhou J, Liu H, and Lü J. Data-driven output consensus tracking control for heterogeneous multi-agent systems with a dynamic leader. IEEE Transactions on Control of Network Systems, 2025, 12(3): 2029−2039 [15] Zhao W, Liu H, Valavanis K P, and Lewis, F L. Fault-tolerant formation control for heterogeneous vehicles via reinforcement learning. IEEE Transactions on Aerospace and Electronic Systems, 2021, 58(4): 2796−2806 [16] Zhao W, Liu H, Wan Y, and Lin Z. Data-driven formation control for multiple heterogeneous vehicles in air-ground coordination. IEEE Transactions on Control of Network Systems, 2022, 9(4): 1851−1862 [17] Liu D, Liu H, Lü J, and Lewis, F L. Time-varying formation of heterogeneous multiagent systems via reinforcement learning subject to switching topologies. IEEE Transactions on Circuits and Systems I: Regular Papers, 2023, 70(6): 2550−2560 [18] Liu H, Ma T, Lewis F L, and Wan Y. Robust formation control for multiple quadrotors with nonlinearities and disturbances. IEEE Transactions on Cybernetics, 2020, 50(4): 1362−1371 [19] Wen G, Lam J, Fu J, and Wang S. Distributed MPC-based robust collision avoidance formation navigation of constrained multiple USVs. IEEE Transactions on Intelligent Vehicles, 2024, 9(1): 1804−1816 [20] 周艳, 温广辉, 郑德智. 基于命令滤波反推方法的水面无人艇容错编队跟踪控制. 指挥与控制学报, 2024, 10(4): 412−419Zhou Y, Wen G, and Zheng D. Fault-tolerant formation tracking control of unmanned surface vehicles based on command filtered backstepping approach. Journal of Command and Control, 2024, 10(4): 412−419 [21] Zhou Y, Zhou J, Wang S, and Wen G. Practical fixed-time attitude consensus tracking for multi-quadrotor systems: A composite learning backstepping approach. IEEE Transactions on Circuits and Systems II: Express Briefs, 2024, 71(6): 3066−3070 [22] Liu Y, Li H, Lu R, Li X, and Lu R. An overview of finite/fixed-time control and its application in engineering systems. IEEE/CAA Journal of Automatica Sinica, 2022, 9(12): 2106−2120 [23] Slotine J and Li W. Applied nonlinear control. Englewood Cliffs, NJ, UAS: Prentice Hall, 1991. [24] Wang D, He H, and Liu D. Adaptive critic nonlinear robust control: A survey. IEEE Transactions on Cybernetics, 2017, 47(10): 3429−3451 [25] Zhao W, Liu H, and Lewis F L. Robust formation control for cooperative underactuated quadrotors via reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(10): 4577−4587 [26] Liu H, Li D, Zuo Z, and Zhong Y. Robust three-loop trajectory tracking control for quadrotors with multiple uncertainties. IEEE Transactions on Industrial Electronics, 2016, 63(4): 2263−2274 -
下载:
计量
- 文章访问数: 27
- HTML全文浏览量: 13
- 被引次数: 0
