Gating Adaptive Repeat Query Framework for Reliable Collaborative Inference with Edge Heterogeneous LLMs

WANG Tengsheng; YU Tao; LI Jihong; ZHENG Guhan; ZHANG Shunqing

doi:10.11999/JEIT260218

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 >

WANG Tengsheng, YU Tao, LI Jihong, ZHENG Guhan, ZHANG Shunqing. Gating Adaptive Repeat Query Framework for Reliable Collaborative Inference with Edge Heterogeneous LLMs[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260218

Citation:

WANG Tengsheng, YU Tao, LI Jihong, ZHENG Guhan, ZHANG Shunqing. Gating Adaptive Repeat Query Framework for Reliable Collaborative Inference with Edge Heterogeneous LLMs[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260218

Citation:

WANG Tengsheng, YU Tao, LI Jihong, ZHENG Guhan, ZHANG Shunqing. Gating Adaptive Repeat Query Framework for Reliable Collaborative Inference with Edge Heterogeneous LLMs[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260218

PDF( 2612 KB)

Gating Adaptive Repeat Query Framework for Reliable Collaborative Inference with Edge Heterogeneous LLMs

doi: 10.11999/JEIT260218 cstr: 32379.14.JEIT260218

1.
School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China
2.
Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, China

Funds: The National Natural Science Foundation of China(62571307), The Science and Technology Commission Foundation of Shanghai(24DP1500703, 24DP1500500), The National Key Research and Development Program of China (2022YFB2902304)

Received Date: 2026-02-28
Accepted Date: 2026-06-24
Rev Recd Date: 2026-06-24

Available Online: 2026-07-04

Abstract

Abstract

Objective Reliable inference at the network edge is indispensable for 6G-enabled ubiquitous AI, yet the deployment of large language models (LLMs) in such environments remains a cornerstone challenge. In resource-constrained edge settings, single-LLM inference often proves unreliable due to knowledge limitations and inherent biases, severely hampering real-world deployment. Collaborative inference leveraging multiple heterogeneous LLMs emerges as a promising remedy to boost robustness, but it introduces nontrivial hurdles under stringent latency and energy budgets, especially when wireless channel conditions and query content vary unpredictably. These challenges include the need for dynamic sequential decision-making for LLM selection and resource allocation, the fundamental paradigm mismatch between bit-level reliability protocols and semantic-level error correction, and the lack of adaptive mechanisms to align and fuse disparate LLM outputs effectively. To fill these critical gaps, this paper presents a novel framework that fundamentally reinterprets collaborative inference as a semantic-driven, closed-loop process, thereby transitioning from conventional bit-retransmission to semantic-retransmission and offering a practical path toward reliable 6G edge intelligence. Methods In response to these critical challenges, we propose the Gating Adaptive Repeat Query (G-ARQ) framework. Its core innovation is the Semantic-Space Alignment and Error-Guided Retransmission (SEMAR) mechanism. SEMAR first aligns the token-level probability distributions from heterogeneous LLMs into a unified semantic space using relative representation, enabling comparable outputs. It then models the collaborative process probabilistically, explicitly capturing error dependencies among models, and uses an Expectation-Maximization (EM) algorithm to infer a latent error direction, which guides the selection of the next LLM for query retransmission, steering it towards outputs orthogonal to previous errors. To jointly optimize the LLM gating and uplink power allocation under communication constraints without requiring explicit system dynamics—often unavailable in practice—we design a black-box trajectory optimizer. This optimizer formulates the sequential decision problem as sampling from a target distribution that encodes dynamic feasibility, optimality, and constraints. It employs a diffusion-based sampling process with a model-guided prior and Monte Carlo estimation to generate near-optimal policy trajectories that satisfy hard latency and energy limits. Results and Discussions To evaluate the practical viability of G-ARQ under realistic edge conditions, simulations are conducted in a scenario with five base stations hosting five heterogeneous 7B-parameter LLMs (Mistral-7B, Vicuna-7B, Nous-Capybala-7B, Gemma-7B, and Llama-2-7B). The user equipment (UE) performs a question-answering task evaluated on a mixed SQuAD and TriviaQA dataset. Component-level evaluations, each designed to isolate the contribution of a single innovation, validate the effectiveness of every key element. The error-guided gating of SEMAR, compared to a Top-k gating baseline, improves accuracy by 0.23 % on average, and its dynamic weight ensemble contributes an additional 0.7 % gain (Fig. 3). The black-box trajectory optimizer, which operates without any explicit channel model, achieves accuracy close to that of the unconstrained model-greedy strategy while ensuring strict latency constraints (Fig. 4). The convergence of the optimizer is verified by tracking the evolution of $ J(\boldsymbol{S}) $ and selection probability over diffusion steps (Fig. 5). System-level performance under varying latency and energy constraints demonstrates that G-ARQ consistently surpasses two baselines: one combining model-greedy selection with Proximal Policy Optimization (PPO) for power optimization, and another combining model-greedy selection with Simulated Annealing for power optimization, both using average output weights. The accuracy improvement is most significant under the most stringent resource limits, reaching up to 2.2 % for $ {K}_{\max }=1 $ and 1.9 % for $ {K}_{\max }=2 $ (Fig. 6, Fig. 7). The framework successfully establishes a Pareto boundary that characterizes the inherent trade-off between inference accuracy and communication latency, providing a valuable design guideline for resource-constrained edge systems and offering actionable insights for real-world deployment. The GARQ-S variant is noted to outperform GARQ-E by avoiding the integration of outputs from previously erroneous models. Conclusions This paper proposed G-ARQ framework, an innovative closed-loop framework that transforms collaborative edge inference into a semantics-guided retransmission process. By introducing SEMAR for error-based alignment and selection of heterogeneous LLMs, and employing a black-box trajectory optimizer for joint model selection and power allocation, the framework achieves up to a 2.2% accuracy improvement under strict resource constraints. The results validate G-ARQ as an effective and practical approach toward reliable and efficient 6G edge intelligence.
- 6G edge intelligence,
- Large language models,
- Collaborative inference,
- Gating adaptive repeat query.

FullText(HTML)

References(16)

References

[1]	CUI Yuanhao, CAO Xiaowen, ZHU Guangxu, et al. Edge perception: Intelligent wireless sensing at network edge[J]. IEEE Communications Magazine, 2025, 63(3): 166–173. doi: 10.1109/MCOM.001.2300660.
[2]	LIU Chang and ZHAO Jun. Resource allocation in large language model integrated 6G vehicular networks[C]. 2024 IEEE 99th Vehicular Technology Conference, Singapore, Singapore, 2024: 1–6. doi: 10.1109/VTC2024-Spring62846.2024.10683673.
[3]	LIN Zheng, QU Guanqiao, CHEN Qiyuan, et al. Pushing large language models to the 6G edge: Vision, challenges, and opportunities[J]. IEEE Communications Magazine, 2025, 63(9): 52–59. doi: 10.1109/MCOM.001.2400764.
[4]	LUO Haoxiang, LIU Yingqiu, ZHANG Ruichen, et al. Toward edge general intelligence with multiple-large language model (Multi-LLM): Architecture, trust, and orchestration[J]. IEEE Transactions on Cognitive Communications and Networking, 2025, 11(6): 3563–3585. doi: 10.1109/TCCN.2025.3612760.
[5]	WANG Lijing, GHOSH D, GONZALEZ DIAZ M T, et al. Wisdom of the ensemble: Improving consistency of deep learning models[C]. Proceedings of the 34th Conference on Neural Information Processing Systems, Vancouver, Canada, 2020: 19750–19761.
[6]	MAZHAR N, ULLAH S A, CHAUHDARY S H, et al. Optimizing age of information in energy-constrained IIoT networks: A reinforcement learning framework[J]. IEEE Internet of Things Journal, 2025, 12(20): 42813–42828. doi: 10.1109/JIOT.2025.3594665.
[7]	AHMED A, AL-DWEIK A, IRAQI Y, et al. Hybrid automatic repeat request (HARQ) in wireless communications systems and standards: A contemporary survey[J]. IEEE Communications Surveys & Tutorials, 2021, 23(4): 2711–2752. doi: 10.1109/COMST.2021.3094401.
[8]	LONG Hang, XIANG Wei, SHEN Shanshan, et al. Analysis of conditional error rate and combining schemes in HARQ[J]. IEEE Transactions on Signal Processing, 2012, 60(5): 2677–2682. doi: 10.1109/TSP.2012.2184100.
[9]	SHLEZINGER N, FARHAN E, MORGENSTERN H, et al. Collaborative inference via ensembles on the edge[C]. ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing, Toronto, Canada, 2021: 8478–8482. doi: 10.1109/ICASSP39728.2021.9414740.
[10]	丁男, 王佳佳, 冀承慧, 等. 边-端协作下基于早期退出机制的深度神经网络动态自适应分区[J]. 电子与信息学报, 2025, 47(10): 4005–4017. doi: 10.11999/JEIT250291. DING Nan, WANG Jiajia, JI Chenghui, et al. Dynamic adaptive partitioning of deep neural networks based on early exit mechanism under edge-end collaboration[J]. Journal of Electronics & Information Technology, 2025, 47(10): 4005–4017. doi: 10.11999/JEIT250291.
[11]	陈阳, 马欢, 姬智, 等. 面向图像恢复任务的语义通信网络能耗优化[J]. 电子与信息学报, 2026, 48(1): 183–190. doi: 10.11999/JEIT250915. CHEN Yang, MA Huan, JI Zhi, et al. Optimization of energy consumption in semantic communication networks for image recovery tasks[J]. Journal of Electronics & Information Technology, 2026, 48(1): 183–190. doi: 10.11999/JEIT250915.
[12]	ZHENG Guhan, NI Qiang, NAVAIE K, et al. Semantic communication in satellite-borne edge cloud network for computation offloading[J]. IEEE Journal on Selected Areas in Communications, 2024, 42(5): 1145–1158. doi: 10.1109/JSAC.2024.3365879.
[13]	林艳, 夏开元, 张一晋. 基于生成对抗网络辅助多智能体强化学习的边缘计算网络联邦切片资源管理[J]. 电子与信息学报, 2025, 47(3): 666–677. doi: 10.11999/JEIT240773. LIN Yan, XIA Kaiyuan, and ZHANG Yijin. Federated slicing resource management in edge computing networks based on GAN-assisted multi-agent reinforcement learning[J]. Journal of Electronics & Information Technology, 2025, 47(3): 666–677. doi: 10.11999/JEIT240773.
[14]	HU Chenbo, YANG Hongjuan, LI Bo, et al. HARQ-aided RSMA for integrated satellite-terrestrial networks[J]. IEEE Transactions on Wireless Communications, 2026, 25: 14987–15003. doi: 10.1109/TWC.2026.3681284.
[15]	HUANG Yichong, FENG Xiaocheng, LI Baohang, et al. Ensemble learning for heterogeneous large language models with deep parallel collaboration[C]. 38th Conference on Neural Information Processing Systems, Vancouver, Canada, 2024: 119838–119860. doi: 10.52202/079017-3808.
[16]	PAN Chaoyi, YI Zeji, SHI Guanya, et al. Model-based diffusion for trajectory optimization[C]. Proceedings of the 38th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2024: 57914–57943.