面向边缘异构大语言模型可靠协作推理的自适应重复查询机制

王腾胜; 余涛; 李济宏; 郑谷寒; 张舜卿

doi:10.11999/JEIT260218

面向边缘异构大语言模型可靠协作推理的自适应重复查询机制

doi: 10.11999/JEIT260218 cstr: 32379.14.JEIT260218

1.
上海大学通信与信息工程学院上海 200444
2.
香港大学电机与电子工程系香港

基金项目: 国家自然科学基金(62571307)，上海市科委基金(24DP1500703, 24DP1500500)，国家重点研发计划(2022YFB2902304)

详细信息

作者简介:
王腾胜：男，硕士生，研究方向为边缘智能、绿色无线通信、无线资源分配

余涛：男，博士后，研究方向为无线网络的节能通信网络、机器学习、深度学习

李济宏：男，博士生，研究方向为网络切片、无线资源分配、节能通信网络

郑谷寒：男，副教授，研究方向为非地面网络、车载网络、语义通信

张舜卿：男，教授，研究方向为节能5G/5G+通信网络、绿色无线网络、异构计算技术

通讯作者:
张舜卿　shunqing@shu.edu.cn

中图分类号: TN929.5
计量
- 文章访问数: 14
- HTML全文浏览量: 4
- PDF下载量: 0
- 被引次数: 0
出版历程
- 收稿日期: 2026-02-28
- 修回日期: 2026-06-24
- 录用日期: 2026-06-24
- 网络出版日期: 2026-07-04

Gating Adaptive Repeat Query Framework for Reliable Collaborative Inference with Edge Heterogeneous LLMs

1.
School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China
2.
Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, China

Funds: The National Natural Science Foundation of China(62571307), The Science and Technology Commission Foundation of Shanghai(24DP1500703, 24DP1500500), The National Key Research and Development Program of China (2022YFB2902304)

摘要

摘要: 在网络边缘部署大语言模型(Large Language Models, LLMs)对泛在人工智能(Artificial Intelligence, AI)至关重要。与单一LLM相比，多LLM协同推理可以显著提高系统的稳健性，但严格的资源约束对可靠推理提出了挑战。本文提出了门控自适应重复查询(Gating Adaptive Repeat Query, G-ARQ)框架，将协同推理转化为语义驱动的闭环过程，增强了系统的健壮性。在G-ARQ中，语义空间对齐和错误引导重传(Semantic-Space Alignment and Error-Guided Retransmission, SEMAR)机制将不同类型的LLM输出对齐到一个统一的语义空间，并使用推理错误信息来指导下一次重复查询LLM的选择，而轨迹优化器在不需要显式系统动态的情况下联合优化LLM门控和资源分配。在SQuAD和TriviaQA混合数据集上的实验表明，在严格的时延和能耗约束下，G-ARQ相比基线方案的准确率最大提高2.2%。G-ARQ框架建立了一个帕累托边界来表征准确率和时延之间的权衡，为可靠的第六代(Sixth-Generation, 6G)边缘智能提供了一条实用的路径。
- 6G边缘智能 /
- 大语言模型 /
- 协同推理 /
- 门控自适应重复查询
Abstract: Objective Reliable inference at the network edge is indispensable for 6G-enabled ubiquitous AI, yet the deployment of large language models (LLMs) in such environments remains a cornerstone challenge. In resource-constrained edge settings, single-LLM inference often proves unreliable due to knowledge limitations and inherent biases, severely hampering real-world deployment. Collaborative inference leveraging multiple heterogeneous LLMs emerges as a promising remedy to boost robustness, but it introduces nontrivial hurdles under stringent latency and energy budgets, especially when wireless channel conditions and query content vary unpredictably. These challenges include the need for dynamic sequential decision-making for LLM selection and resource allocation, the fundamental paradigm mismatch between bit-level reliability protocols and semantic-level error correction, and the lack of adaptive mechanisms to align and fuse disparate LLM outputs effectively. To fill these critical gaps, this paper presents a novel framework that fundamentally reinterprets collaborative inference as a semantic-driven, closed-loop process, thereby transitioning from conventional bit-retransmission to semantic-retransmission and offering a practical path toward reliable 6G edge intelligence. Methods In response to these critical challenges, we propose the Gating Adaptive Repeat Query (G-ARQ) framework. Its core innovation is the Semantic-Space Alignment and Error-Guided Retransmission (SEMAR) mechanism. SEMAR first aligns the token-level probability distributions from heterogeneous LLMs into a unified semantic space using relative representation, enabling comparable outputs. It then models the collaborative process probabilistically, explicitly capturing error dependencies among models, and uses an Expectation-Maximization (EM) algorithm to infer a latent error direction, which guides the selection of the next LLM for query retransmission, steering it towards outputs orthogonal to previous errors. To jointly optimize the LLM gating and uplink power allocation under communication constraints without requiring explicit system dynamics—often unavailable in practice—we design a black-box trajectory optimizer. This optimizer formulates the sequential decision problem as sampling from a target distribution that encodes dynamic feasibility, optimality, and constraints. It employs a diffusion-based sampling process with a model-guided prior and Monte Carlo estimation to generate near-optimal policy trajectories that satisfy hard latency and energy limits. Results and Discussions To evaluate the practical viability of G-ARQ under realistic edge conditions, simulations are conducted in a scenario with five base stations hosting five heterogeneous 7B-parameter LLMs (Mistral-7B, Vicuna-7B, Nous-Capybala-7B, Gemma-7B, and Llama-2-7B). The user equipment (UE) performs a question-answering task evaluated on a mixed SQuAD and TriviaQA dataset. Component-level evaluations, each designed to isolate the contribution of a single innovation, validate the effectiveness of every key element. The error-guided gating of SEMAR, compared to a Top-k gating baseline, improves accuracy by 0.23 % on average, and its dynamic weight ensemble contributes an additional 0.7 % gain (Fig. 3). The black-box trajectory optimizer, which operates without any explicit channel model, achieves accuracy close to that of the unconstrained model-greedy strategy while ensuring strict latency constraints (Fig. 4). The convergence of the optimizer is verified by tracking the evolution of $ J(\boldsymbol{S}) $ and selection probability over diffusion steps (Fig. 5). System-level performance under varying latency and energy constraints demonstrates that G-ARQ consistently surpasses two baselines: one combining model-greedy selection with Proximal Policy Optimization (PPO) for power optimization, and another combining model-greedy selection with Simulated Annealing for power optimization, both using average output weights. The accuracy improvement is most significant under the most stringent resource limits, reaching up to 2.2 % for $ {K}_{\max }=1 $ and 1.9 % for $ {K}_{\max }=2 $ (Fig. 6, Fig. 7). The framework successfully establishes a Pareto boundary that characterizes the inherent trade-off between inference accuracy and communication latency, providing a valuable design guideline for resource-constrained edge systems and offering actionable insights for real-world deployment. The GARQ-S variant is noted to outperform GARQ-E by avoiding the integration of outputs from previously erroneous models. Conclusions This paper proposed G-ARQ framework, an innovative closed-loop framework that transforms collaborative edge inference into a semantics-guided retransmission process. By introducing SEMAR for error-based alignment and selection of heterogeneous LLMs, and employing a black-box trajectory optimizer for joint model selection and power allocation, the framework achieves up to a 2.2% accuracy improvement under strict resource constraints. The results validate G-ARQ as an effective and practical approach toward reliable and efficient 6G edge intelligence.
- 6G edge intelligence /
- Large language models /
- Collaborative inference /
- Gating adaptive repeat query.

HTML全文

图 1 G-ARQ系统架构

下载: 全尺寸图片幻灯片

图 2 所提算法框图

下载: 全尺寸图片幻灯片

图 3 评估SEMAR有效性的实验结果

下载: 全尺寸图片幻灯片

图 4 不同方案下的准确率、时延随$ K $的变化

下载: 全尺寸图片幻灯片

图 5 所提优化算法的收敛性

下载: 全尺寸图片幻灯片

图 6 不同方案和$ {K}_{\max } $下的准确率随最大时延约束的变化

下载: 全尺寸图片幻灯片

图 7 不同方案和$ {K}_{\max } $下的准确率随最大能耗约束的变化

下载: 全尺寸图片幻灯片

表 1 HARQ与GARQ的区别

对比维度	HARQ	GARQ
可靠性目标	关注比特级可靠性，通过冗余比特重传保证数据正确解码。	关注任务级语义可靠性，通过重新查询合适的LLM提升最终任务完成质量。
错误判定单元	以传输块为判定单元，ACK/NACK通常由CRC校验产生， NACK表示解码失败。	以LLM语义输出为判定单元，ACK/NACK表示输出是否满足任务需求，NACK表示语义失败。
资源优化对象	单次链路或有限重传中的编码率、功率、重传次数等链路资源。	联合优化多轮查询中的LLM选择、通信资源分配和语义融合权重。

下载: 导出CSV

表 2 仿真参数设置

参数名称	参数值
基站个数($ {N}_{\text{BS}} $)	5
带宽($ B $) 最大发射功率($ {p}_{\max } $) 路径损耗	20 MHz 27 dBm 35+35lg($ d $)
噪声功率谱密度($ {\sigma }^{2} $)	–174 dBm/Hz
扩散步数($ N $)	1000
蒙特卡洛采样样本数($ M $)	8192
最大重查询次数($ {K}_{\max } $)	2

下载: 导出CSV

表 3 在不同数据集下的准确率比较

方案	数据集
方案	GSM8K	TriviaQA	ARC-C	SquAD
基线1	48.65	70.20	72.10	80.70
基线2	48.05	69.80	71.90	80.30
GARQ-E	51.20	71.30	72.35	80.90
GARQ-S	50.30	71.40	74.32	81.80

下载: 导出CSV

表 4 不同方案的运行时间比较

方案	GARQ	基线1	基线2
运行时间/秒	4.287	16.415	55.621

下载: 导出CSV

参考文献(16)

[1]	CUI Yuanhao, CAO Xiaowen, ZHU Guangxu, et al. Edge perception: Intelligent wireless sensing at network edge[J]. IEEE Communications Magazine, 2025, 63(3): 166–173. doi: 10.1109/MCOM.001.2300660.
[2]	LIU Chang and ZHAO Jun. Resource allocation in large language model integrated 6G vehicular networks[C]. 2024 IEEE 99th Vehicular Technology Conference, Singapore, Singapore, 2024: 1–6. doi: 10.1109/VTC2024-Spring62846.2024.10683673.
[3]	LIN Zheng, QU Guanqiao, CHEN Qiyuan, et al. Pushing large language models to the 6G edge: Vision, challenges, and opportunities[J]. IEEE Communications Magazine, 2025, 63(9): 52–59. doi: 10.1109/MCOM.001.2400764.
[4]	LUO Haoxiang, LIU Yingqiu, ZHANG Ruichen, et al. Toward edge general intelligence with multiple-large language model (Multi-LLM): Architecture, trust, and orchestration[J]. IEEE Transactions on Cognitive Communications and Networking, 2025, 11(6): 3563–3585. doi: 10.1109/TCCN.2025.3612760.
[5]	WANG Lijing, GHOSH D, GONZALEZ DIAZ M T, et al. Wisdom of the ensemble: Improving consistency of deep learning models[C]. Proceedings of the 34th Conference on Neural Information Processing Systems, Vancouver, Canada, 2020: 19750–19761.
[6]	MAZHAR N, ULLAH S A, CHAUHDARY S H, et al. Optimizing age of information in energy-constrained IIoT networks: A reinforcement learning framework[J]. IEEE Internet of Things Journal, 2025, 12(20): 42813–42828. doi: 10.1109/JIOT.2025.3594665.
[7]	AHMED A, AL-DWEIK A, IRAQI Y, et al. Hybrid automatic repeat request (HARQ) in wireless communications systems and standards: A contemporary survey[J]. IEEE Communications Surveys & Tutorials, 2021, 23(4): 2711–2752. doi: 10.1109/COMST.2021.3094401.
[8]	LONG Hang, XIANG Wei, SHEN Shanshan, et al. Analysis of conditional error rate and combining schemes in HARQ[J]. IEEE Transactions on Signal Processing, 2012, 60(5): 2677–2682. doi: 10.1109/TSP.2012.2184100.
[9]	SHLEZINGER N, FARHAN E, MORGENSTERN H, et al. Collaborative inference via ensembles on the edge[C]. ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing, Toronto, Canada, 2021: 8478–8482. doi: 10.1109/ICASSP39728.2021.9414740.
[10]	丁男, 王佳佳, 冀承慧, 等. 边-端协作下基于早期退出机制的深度神经网络动态自适应分区[J]. 电子与信息学报, 2025, 47(10): 4005–4017. doi: 10.11999/JEIT250291. DING Nan, WANG Jiajia, JI Chenghui, et al. Dynamic adaptive partitioning of deep neural networks based on early exit mechanism under edge-end collaboration[J]. Journal of Electronics & Information Technology, 2025, 47(10): 4005–4017. doi: 10.11999/JEIT250291.
[11]	陈阳, 马欢, 姬智, 等. 面向图像恢复任务的语义通信网络能耗优化[J]. 电子与信息学报, 2026, 48(1): 183–190. doi: 10.11999/JEIT250915. CHEN Yang, MA Huan, JI Zhi, et al. Optimization of energy consumption in semantic communication networks for image recovery tasks[J]. Journal of Electronics & Information Technology, 2026, 48(1): 183–190. doi: 10.11999/JEIT250915.
[12]	ZHENG Guhan, NI Qiang, NAVAIE K, et al. Semantic communication in satellite-borne edge cloud network for computation offloading[J]. IEEE Journal on Selected Areas in Communications, 2024, 42(5): 1145–1158. doi: 10.1109/JSAC.2024.3365879.
[13]	林艳, 夏开元, 张一晋. 基于生成对抗网络辅助多智能体强化学习的边缘计算网络联邦切片资源管理[J]. 电子与信息学报, 2025, 47(3): 666–677. doi: 10.11999/JEIT240773. LIN Yan, XIA Kaiyuan, and ZHANG Yijin. Federated slicing resource management in edge computing networks based on GAN-assisted multi-agent reinforcement learning[J]. Journal of Electronics & Information Technology, 2025, 47(3): 666–677. doi: 10.11999/JEIT240773.
[14]	HU Chenbo, YANG Hongjuan, LI Bo, et al. HARQ-aided RSMA for integrated satellite-terrestrial networks[J]. IEEE Transactions on Wireless Communications, 2026, 25: 14987–15003. doi: 10.1109/TWC.2026.3681284.
[15]	HUANG Yichong, FENG Xiaocheng, LI Baohang, et al. Ensemble learning for heterogeneous large language models with deep parallel collaboration[C]. 38th Conference on Neural Information Processing Systems, Vancouver, Canada, 2024: 119838–119860. doi: 10.52202/079017-3808.
[16]	PAN Chaoyi, YI Zeji, SHI Guanya, et al. Model-based diffusion for trajectory optimization[C]. Proceedings of the 38th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2024: 57914–57943.