A Multi-agent Active Inference-based Collaborative Decision-making Method for Space-Air-Ground Integrated Networks
-
摘要: 针对空天地一体化网络(SAGIN)中局部观测不完备、拓扑时变和多业务资源耦合的协同决策难题，该文提出基于集体主动推理(CAI)的生智-传智一体化方法。该方法以集体变分自由能最小化推导精度加权语义信念更新律及几何收敛界，以垂直斯塔克尔伯格博弈和水平广义纳什均衡协调共享资源，并结合差分隐私联邦学习、模型无关元学习和知识蒸馏实现跨域适配与轻量部署。开放无线接入网系统级仿真中，满载吞吐量为93.2 Mb/s，任务调度成功率为0.92，95分位时延为20.5 ms；分布外扰动后23个周期恢复。Top-10%稀疏上传使通信量由321.79×103降至34.58×103个标量，学生模型由1 050 kB压缩至160 kB、推理时延由21 ms降至15 ms。结果表明，所提方法在既定仿真条件下兼顾语义共识、资源协调、扰动恢复与受限节点部署。Abstract:
Objective Space-Air-Ground Integrated Networks (SAGINs) combine satellites, aerial platforms, terrestrial access points, and edge nodes to support heterogeneous services in highly dynamic environments. Collaborative decision-making is challenging because observations are local and delayed, network topology varies over time, raw-data exchange is restricted, and spectrum, power, and computing resources are coupled. Existing Multi-Agent Reinforcement Learning (MARL) and game-theoretic methods do not jointly address uncertainty-aware semantic consensus, hierarchical resource allocation, cross-domain adaptation, and lightweight deployment. This study develops a Collective Active Inference (CAI)-based integrated intelligence generation and transmission loop. Agents exchange inferred beliefs rather than raw observations, consensus beliefs guide resource decisions, and learned models are adapted and compressed for resource-constrained satellite and edge nodes. The method targets time-varying, partially observable SAGINs and is evaluated through an Open Radio Access Network (O-RAN) system-level simulation rather than an operational deployment. Methods The problem is formulated as a time-varying decentralized partially observable Markov decision process over channel, queue, congestion, interference, and node states. This formulation captures decentralized decisions under incomplete information and intermittent connectivity. Multi-agent cognition is modeled by minimizing Collective Variational Free Energy (CVFE). A likelihood term preserves local observation fidelity, while a Kullback-Leibler divergence term penalizes disagreement between neighboring beliefs. A Gaussian precision-weighted belief update assigns greater weight to more reliable beliefs. Symmetric Metropolis weights produce a doubly stochastic mixing matrix. For a fixed connected graph, the consensus error decreases geometrically according to the second-largest eigenvalue modulus. This result extends to intermittent links under B-joint connectivity. Agents exchange only belief means and covariance matrices and act on converged beliefs (Algorithm 1). A vertical-horizontal hierarchical game maps consensus beliefs to resource allocations. Vertically, the operator acts as the Stackelberg leader, and service slices respond to prices. Horizontally, the slices reach a Generalized Nash Equilibrium (GNE) under shared spectrum, power, and Service Level Agreement (SLA) constraints. Cross-domain adaptation combines Differentially Private Federated Learning (DP-FL), top-10% gradient sparsification, Model-Agnostic Meta-Learning (MAML), and knowledge distillation. The distillation temperature is 4, and the supervised and distillation loss weights are 0.3 and 0.7, respectively. Evaluation is conducted using an OMNeT++ O-RAN simulator with 12 coverage nodes, 19 underlay nodes, 72 links, a 28 GHz carrier, and 100 MHz bandwidth. Concurrent Ultra-Reliable and Low-Latency Communications (URLLC), enhanced Mobile BroadBand (eMBB), and massive Machine-Type Communications (mMTC) traffic is simulated for 10 000 control periods. The network load factor increases from 0.2 to 1.0. Frequency-sweeping interference and traffic bursts are introduced to test Out-Of-Distribution (OOD) recovery. All baselines use the same observations, action space, constraints, training budget, and random-seed set. Reported values are averaged over at least 10 independent runs with 95% confidence intervals. Results and Discussions At full load, the proposed method achieves a throughput of 93.2 Mb/s, compared with 81.8, 88.5, and 78.4 Mb/s for Symmetry-Informed MARL (SI-MARL), Quantum MARL (QMARL), and Graph Attention Network-based MARL (GAT-MARL), respectively. Its task-scheduling success rate is 0.92. The 95th-percentile latency is 20.5 ms, compared with 31.2, 26.4, and 33.0 ms for the three baselines, and the SLA violation rate is approximately 0.081 ( Fig. 3 ). After an OOD disturbance, the Key Performance Indicator (KPI) retention rate is approximately 0.83, and recovery to the 95% threshold requires 23 control periods, compared with 35 and 41 periods for QMARL and GAT-MARL, respectively (Fig. 4 ). In the resource-competition scenario, the mean task-scheduling success rate is 0.92, with a 95% confidence interval of [0.90, 0.94]. The corresponding values for three hierarchical-game or federated-game baselines range from 0.80 to 0.85 (Fig. 5). With 50 agents and an update dimension of 80, top-10% gradient sparsification reduces the number of transmitted scalars from 321.79 × 103 to 34.58 × 103, an 89.3% reduction. The number of convergence rounds increases from 58.84 to 71.26, or by 21.1% (Table 1 ). For few-shot relation inference, the F1 score is 0.71 with 20 labeled samples and 0.98 with 80 labeled samples. Approximately 48 labeled samples are required to reach an F1 score of 0.90, compared with 75, 90, and 120 samples for the baselines (Fig. 6 ). Knowledge distillation reduces the model size from 1 050 KB to 160 KB and the inference latency from 21 ms to 15 ms. The student model retains a task success rate of 91% and a path efficiency of 93%, compared with 95% and 97%, respectively, for the teacher model. Removing environment-domain information causes the largest loss in path efficiency (Table 2 ).Conclusions The proposed CAI-based method forms a closed loop that combines precision-weighted semantic consensus, hierarchical resource coordination, cross-domain adaptation, and lightweight deployment. The convergence analysis relates consensus speed to graph connectivity. Under the stated simulation settings, the O-RAN evaluation shows higher throughput and task-scheduling success rates, lower 95th-percentile latency and SLA violation rates, and faster disturbance recovery. Top-10% gradient sparsification reduces communication overhead, whereas knowledge distillation reduces model size and inference latency. The current evaluation is limited by simulated channels, idealized observation quality, synchronous updates, and modeled prior distributions that may differ from operational conditions. These results therefore do not guarantee performance across arbitrary operational networks. Further validation using measured channels, hardware-in-the-loop tests, and field experiments is required. -
1 基于CVFE的分布式多智能体生智共识
输入：通信图$ G=(\mathcal{N},{E}_{{\mathrm{comm}}}) $、本地观测序列$ {\boldsymbol{o}}_{t}^{i} $、 初始信念$ {Q}_{0i} $、耦合强度$ {\lambda }_{ij} $、迭代上限$ {T}_{{\mathrm{iter}}} $ 输出：达成共识的信念参数$ \{{\boldsymbol{\mu}} _{i}^{*},{{\boldsymbol{\varSigma}} }_{i}^{*}\} $ 与动作$ {a}_{i} $ 1: for t = 1, 2, ··· do 2: for each agent i ∈ $ {\mathcal{N}} $ in parallel do 3: 基于$ \boldsymbol{o}_t^i $进行本地变分推断，得到($ {\boldsymbol{\mu}} _{{i}}^{0} $, ${\boldsymbol{\varSigma}}_{i}^{0} $) 4: for $ \tau $ = 0,1,···,$ {{T}}_{\mathrm{iter}-1} $ do // 共识迭代 5: 广播($ \boldsymbol{\mu}_i^{\tau} $,$ {\boldsymbol{\varSigma}}_{i}^{\tau } $)至$ {\mathcal{N}}_{i} $；接收邻居参数 6: 按式(6)更新 $ \boldsymbol{\mu}_i(\tau+1) $ 7: if $ \| {\boldsymbol{\mu}}_{i}^{\tau +1}-{{\boldsymbol{\mu}}}_{i}(\tau )\| <\varepsilon $ then break 8: end for 9: 基于$ {\boldsymbol{\mu}} _{i}^{*} $选择动作$ {a}_{i}\sim {\pi }_{i}(\cdot |{\boldsymbol{\mu}}_{i}^{*}) $，执行并接收奖励$ {r}_{i} $ 10: end for 11: end for 
下载: 导出CSV
表 1 联邦学习通信开销与收敛轮数对比
智能体数 维度 全量通信
(×103)稀疏通信
(×103)全量轮数 稀疏轮数 10 20 4.06 0.53 33.69 40.26 10 50 25.19 2.82 39.09 46.63 10 80 64.36 6.92 44.49 53.00 30 20 12.18 1.58 42.01 50.79 30 50 75.58 8.46 47.41 57.16 30 80 193.07 20.75 52.81 63.53 50 20 20.29 2.63 48.04 58.51 50 50 126.00 14.10 53.44 64.88 50 80 321.79 34.58 58.84 71.26
下载: 导出CSV
表 2 知识蒸馏后的轻量部署效果及消融实验
方法 路径效率(%) 成功率(%) 大小(kB) 时延(ms) 深度强化学习教师 97.0 95.0 1 050 21 所提方法(学生) 93.0 91.0 160 15 缺环境域消融 82.0 77.0 160 16 缺网络域消融 85.0 73.0 165 18 缺行为域消融 88.0 79.0 168 17
下载: 导出CSV
-
[1] GUO Hongzhi, LI Jingyi, LIU Jiajia, et al. A survey on space-air-ground-sea integrated network security in 6G[J]. IEEE Communications Surveys & Tutorials, 2022, 24(1): 53–87. doi: 10.1109/COMST.2021.3131332. [2] 王雪, 孟姝宇, 钱志鸿. 面向6G全域融合的智能接入关键技术综述[J]. 电子与信息学报, 2024, 46(5): 1613–1631. doi: 10.11999/JEIT231224.WANG Xue, MENG Shuyu, and QIAN Zhihong. An overview of key technologies for intelligent access toward 6G full-domain convergence[J]. Journal of Electronics & Information Technology, 2024, 46(5): 1613–1631. doi: 10.11999/JEIT231224. [3] 林佳琦, 钱琪杰, 钟旭东, 等. 面向6G的知识驱动“自智”网络架构[J]. 通信学报, 2025, 46(10): 40–62. doi: 10.11959/j.issn.1000-436x.2025159.LIN Jiaqi, QIAN Qijie, ZHONG Xudong, et al. Knowledge-driven “self-intelligent” network architecture for 6G[J]. Journal on Communications, 2025, 46(10): 40–62. doi: 10.11959/j.issn.1000-436x.2025159. [4] SHI Rongye, YU Xin, WANG Yandong, et al. Symmetry-Informed MARL: A decentralized and cooperative UAV swarm control approach for communication coverage[J]. IEEE Transactions on Mobile Computing, 2025, 24(9): 8039–8056. doi: 10.1109/TMC.2025.3553285. [5] KIM G S, CHO Y, CHUNG J, et al. Quantum multi-agent reinforcement learning for cooperative mobile access in space-air-ground integrated networks[J]. IEEE Transactions on Mobile Computing, 2026, 25(1): 1200–1218. doi: 10.1109/TMC.2025.3599683. [6] AO Tianyong, LI Haoqiang, ZHANG Kaixin, et al. Heterogeneous UAVs trajectory optimization for post-disaster target search based on MARL with graph attention network[J]. IEEE Transactions on Vehicular Technology, 2026, 75(1): 1412–1426. doi: 10.1109/TVT.2025.3594534. [7] ZHANG Jianshu and WU Xiaofu. Cooperative jamming over DRL-based frequency hopping wireless communications: A one-leader multi-follower Stackelberg game approach[J]. IEEE Transactions on Information Forensics and Security, 2025, 20: 9220–9234. doi: 10.1109/TIFS.2025.3604229. [8] LIN Xin, LIU Aijun, HAN Chen, et al. Intelligent adaptive MIMO transmission for nonstationary communication environment: A deep reinforcement learning approach[J]. IEEE Transactions on Communications, 2025, 73(8): 5965–5979. doi: 10.1109/TCOMM.2025.3529263. [9] LI Peixuan, WANG Yichen, WANG Zhangnan, et al. Joint task offloading and resource allocation strategy for hybrid MEC-enabled LEO satellite networks: A hierarchical game approach[J]. IEEE Transactions on Communications, 2025, 73(5): 3150–3166. doi: 10.1109/TCOMM.2024.3478111. [10] CHEN Tianjiao, WANG Xiaoyun, HUA Meihui, et al. Incentive-based task offloading for digital twins in 6G native artificial intelligence networks: A learning approach[J]. Frontiers of Information Technology & Electronic Engineering, 2025, 26(2): 214–229. doi: 10.1631/FITEE.2400240. [11] WANG Kaidi, MA Yi, MASHHADI M B, et al. Convergence acceleration in wireless federated learning: A Stackelberg game approach[J]. IEEE Transactions on Vehicular Technology, 2025, 74(1): 714–729. doi: 10.1109/TVT.2024.3452933. [12] 张鸿, 廖彧歆, 王汝言, 等. 面向密集场景的空天地网络资源分配算法[J]. 电子与信息学报, 2024, 46(5): 1968–1976. doi: 10.11999/JEIT231086.ZHANG Hong, LIAO Yuxin, WANG Ruyan, et al. Resource allocation algorithm of space-air-ground integrated network for dense scenarios[J]. Journal of Electronics & Information Technology, 2024, 46(5): 1968–1976. doi: 10.11999/JEIT231086. [13] 林艳, 夏开元, 张一晋. 基于生成对抗网络辅助多智能体强化学习的边缘计算网络联邦切片资源管理[J]. 电子与信息学报, 2025, 47(3): 666–677. doi: 10.11999/JEIT240773.LIN Yan, XIA Kaiyuan, and ZHANG Yijin. Federated slicing resource management in edge computing networks based on GAN-assisted multi-agent reinforcement learning[J]. Journal of Electronics & Information Technology, 2025, 47(3): 666–677. doi: 10.11999/JEIT240773. [14] REZAZADEH F, CHERGUI H, and MANGUES-BAFALLUY J. Explanation-guided deep reinforcement learning for trustworthy 6G RAN slicing[C]. 2023 IEEE International Conference on Communications Workshops (ICC Workshops), Rome, Italy, 2023: 1026–1031. doi: 10.1109/ICCWorkshops57953.2023.10283684. [15] HONG J, VAN TU N, and HONG J W K. A comprehensive survey on LLM-based network management and operations[J]. International Journal of Network Management, 2025, 35(6): e70029. doi: 10.1002/nem.70029. [16] SUBRAMANIAN A, BHATTACHARJEE A, GUPTA S K, et al. A neuro-symbolic approach to multi-agent RL for interpretability and probabilistic decision making[C]. AAAI Spring Symposium on User-Aligned Assessment of Adaptive AI Systems, Stanford, USA, 2024. [17] FRISTON K, FITZGERALD T, RIGOLI F, et al. Active inference: A process theory[J]. Neural Computation, 2017, 29(1): 1–49. doi: 10.1162/NECO_a_00912. [18] NEDIC A and OZDAGLAR A. Distributed subgradient methods for multi-agent optimization[J]. IEEE Transactions on Automatic Control, 2009, 54(1): 48–61. doi: 10.1109/TAC.2008.2009515. [19] FINN C, ABBEEL P, and LEVINE S. Model-agnostic meta-learning for fast adaptation of deep networks[C]. The 34th International Conference on Machine Learning, Sydney, Australia, 2017: 1126–1135. [20] HINTON G, VINYALS O, and DEAN J. Distilling the knowledge in a neural network[EB/OL]. https://arxiv.org/abs/1503.02531, 2015. doi: 10.48550/arXiv.1503.02531. -
下载:
图(6) / 表(3)
计量
- 文章访问数: 295
- HTML全文浏览量: 52
- PDF下载量: 34
- 被引次数: 0
