A Reinforcement Learning Driven Power Allocation Algorithm for Collocated MIMO Radar

HUANG Jieyu; XIE Junwei; ZHANG Haowei; FENG Weike; HAN Weihang

doi:10.11999/JEIT260695

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 >

HUANG Jieyu, XIE Junwei, ZHANG Haowei, FENG Weike, HAN Weihang. A Reinforcement Learning Driven Power Allocation Algorithm for Collocated MIMO Radar[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260695

Citation:

HUANG Jieyu, XIE Junwei, ZHANG Haowei, FENG Weike, HAN Weihang. A Reinforcement Learning Driven Power Allocation Algorithm for Collocated MIMO Radar[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260695

Citation:

PDF( 3336 KB)

A Reinforcement Learning Driven Power Allocation Algorithm for Collocated MIMO Radar

doi: 10.11999/JEIT260695 cstr: 32379.14.JEIT260695

Air Force Engineering University Air and Missile Defense College, Xi’an 710051, China

Funds: National Natural Science Foundation of China (62571544), Innovation Capability Support Program of Shaanxi (2025ZC-KJXX-81), Research Program Project of Youth Innovation Team of Shaanxi Provincial Education Department (24JP221), Natural Science Foundation of Xi’an (26ZRKX00090)

Received Date: 2026-05-28
Accepted Date: 2026-07-03
Rev Recd Date: 2026-07-02

Available Online: 2026-07-14

Abstract

Abstract

Objective Traditional optimization based on power allocation algorithm for collocated MIMO radar has two fundamental limitations. First, it optimizes tracking performance only for the next time step, lacking a global view across the entire time horizon. This myopic strategy cannot achieve optimal multiple target tracking accuracy over a long duration, especially when target trajectories vary significantly. Second, their iterative solving process involves nonlinear constrained optimization, which incurs high computational complexity. In dynamic battlefield environments where target states change rapidly, such algorithms fail to meet real-time requirements. To address these issues, this paper proposes a reinforcement learning (RL) driven power allocation algorithm. Unlike traditional methods, the proposed approach formulates the problem as a Markov decision process that maximizes long-term cumulative rewards. The algorithm adaptively allocates limited power resources among multiple beams based on the current system state, balancing immediate tracking performance and future gains. Methods The posterior Cramér-Rao lower bound (PCRLB) is employed to quantify the theoretical tracking error lower bound for each target. The state space is constructed by combining the motion states (position and velocity) of all targets and normalized PCRLB from the previous allocation. The action space consists of discrete transmit power levels for each beam, subject to the total power budget and beam power limits. All feasible power allocation vectors are enumerated and encoded to reduce dimensionality. The reward function is defined as the negative weighted sum of normalized PCRLB, encouraging the agent to minimize tracking errors. The power allocation process is formulated as a Markov decision process (MDP). The Dueling Double Deep Q-Network (D3QN) algorithm is adopted to solve this MDP. D3QN integrates three key enhancements: (1) a double network training framework (decision Q-network and target Q-network) to stabilize learning; (2) a dueling architecture that decomposes the Q-value into state value and action advantage functions, improving action discrimination; (3) off-policy learning with experience replay, enabling efficient use of historical trajectories. The $ \varepsilon \text{-greedy} $strategy is used for exploration, with epsilon decaying over episodes. After offline training, the learned network outputs power allocation decisions in real-time given the current system state, without any iterative optimization. Results and Discussions Simulations are conducted with three targets following constant velocity models. Fixed power allocation yields the lowest tracking accuracy due to inefficient resource utilization. The traditional optimization method, which minimizes the immediate tracking error, achieves moderate accuracy but remains myopic. For D3QN with discount factor $ \gamma =0 $, the performance is nearly identical to the traditional method. In contrast, D3QN with $ \gamma =0.99 $ achieves significantly better full time horizon accuracy. Power allocation patterns reveal that the D3QN with $ \gamma =0.99 $ preemptively allocates more power to distant and low signal to noise ratio (SNR) targets earlier, while reducing redundant power to close and high SNR targets. The training curves show that $ \gamma =0.99 $ achieves a higher steady state cumulative reward, although with more oscillations due to the complexity of estimating future returns. Moreover, the trained D3QN network outputs decisions instantaneously, whereas traditional optimization requires solving a constrained optimization problem at each time step, providing a clear real-time advantage. Conclusions This paper proposes a RL driven power allocation algorithm for collocated MIMO radar multiple target tracking that overcomes the myopic and computationally intensive limitations of traditional optimization methods. The algorithm uses PCRLB to construct state and reward functions, models the allocation process as an MDP, and solves it using the D3QN algorithm. Simulation results demonstrate that the approach based on D3QN with a suitable discount factor ($ \gamma =0.99 $) significantly improves full time horizon target tracking accuracy. The improvement stems from the agent’s ability to learn a long-term optimal policy that preemptively allocates resources to future challenging targets. Furthermore, the trained network enables real-time decision, substantially reducing computational latency from iterative solving to instantaneous forward propagation. This work provides a new approach for intelligent radar resource management in complex battlefield environments.
- Reinforcement learning (RL),
- Collocated MIMO radar,
- Power resource allocation,
- Dueling Double Deep Q-Network (D3QN) algorithm

FullText(HTML)

References(15)

References

[1]	何子述, 程子扬, 李军, 等. 集中式MIMO雷达研究综述[J]. 雷达学报, 2022, 11(5): 805–829. doi: 10.12000/JR22128. HE Zishu, CHENG Ziyang, LI Jun, et al. A survey of collocated MIMO radar[J]. Journal of Radars, 2022, 11(5): 805–829. doi: 10.12000/JR22128.
[2]	严俊坤, 陈林, 刘宏伟, 等. 基于机会约束的MIMO雷达多波束稳健功率分配算法[J]. 电子学报, 2019, 47(6): 1230–1235. doi: 10.3969/j.issn.0372-2112.2019.06.007. YAN Junkun, CHEN Lin, LIU Hongwei, et al. Chance constrained based robust multibeam power allocation algorithm for MIMO radar[J]. Acta Electronica Sinica, 2019, 47(6): 1230–1235. doi: 10.3969/j.issn.0372-2112.2019.06.007.
[3]	李正杰, 谢军伟, 张浩为, 等. 一种低截获背景下的集中式MIMO雷达快速功率分配算法[J]. 雷达学报, 2023, 12(3): 602–615. doi: 10.12000/JR22203. LI Zhengjie, XIE Junwei, ZHANG Haowei, et al. A fast power allocation algorithm in a collocated MIMO radar under low interception backgrounds[J]. Journal of Radars, 2023, 12(3): 602–615. doi: 10.12000/JR22203.
[4]	时晨光, 丁琳涛, 汪飞, 等. 面向射频隐身的组网雷达多目标跟踪下射频辐射资源优化分配算法[J]. 电子与信息学报, 2021, 43(3): 539–546. doi: 10.11999/JEIT200636. SHI Chenguang, DING Lintao, WANG Fei, et al. Radio frequency stealth-based optimal radio frequency resource allocation algorithm for multiple-target tracking in radar network[J]. Journal of Electronics & Information Technology, 2021, 43(3): 539–546. doi: 10.11999/JEIT200636.
[5]	黄洁瑜, 张浩为, 谢军伟, 等. 一种面向隐身目标跟踪的雷达组网系统资源优化分配算法[J]. 北京航空航天大学学报, 2026, 52(2): 470–481. doi: 10.13700/j.bh.1001-5965.2023.0782. HUANG Jieyu, ZHANG Haowei, XIE Junwei, et al. A resource optimization allocation algorithm for radar networked system for stealth target tracking[J]. Journal of Beijing University of Aeronautics and Astronautics, 2026, 52(2): 470–481. doi: 10.13700/j.bh.1001-5965.2023.0782.
[6]	YUAN Ye, YI Wei, HOSEINNEZHAD R, et al. Robust power allocation for resource-aware multi-target tracking with colocated MIMO radars[J]. IEEE Transactions on Signal Processing, 2021, 69: 443–458. doi: 10.1109/TSP.2020.3047519.
[7]	ZHANG Haowei, SHI Junpeng, ZHANG Qiliang, et al. Antenna selection for target tracking in collocated MIMO radar[J]. IEEE Transactions on Aerospace and Electronic Systems, 2021, 57(1): 423–436. doi: 10.1109/TAES.2020.3031767.
[8]	丁梓航, 谢军伟, 齐铖. 基于强化学习的频控阵-多输入多输出雷达发射功率分配方法[J]. 电子与信息学报, 2023, 45(2): 550–557. doi: 10.11999/JEIT211555. DING Zihang, XIE Junwei, and QI Cheng. Transmit power allocation method of frequency diverse array-multi input and multi output radar based on reinforcement learning[J]. Journal of Electronics & Information Technology, 2023, 45(2): 550–557. doi: 10.11999/JEIT211555.
[9]	张鹏, 严俊坤, 高畅, 等. 动态电磁环境下多功能雷达一体化发射资源管理方案[J]. 雷达学报, 2025, 14(2): 456–469. doi: 10.12000/JR24230. ZHANG Peng, YAN Junkun, GAO Chang, et al. Integrated transmission resource management scheme for multifunctional radars in dynamic electromagnetic environments[J]. Journal of Radars, 2025, 14(2): 456–469. doi: 10.12000/JR24230.
[10]	LU Ziyang and GURSOY M C. Resource allocation for multi-target radar tracking via constrained deep reinforcement learning[J]. IEEE Transactions on Cognitive Communications and Networking, 2023, 9(6): 1677–1690. doi: 10.1109/TCCN.2023.3304634.
[11]	易伟, 袁野, 刘光宏, 等. 多雷达协同探测技术研究进展: 认知跟踪与资源调度算法[J]. 雷达学报, 2023, 12(3): 471–499. doi: 10.12000/JR23036. YI Wei, YUAN Ye, LIU Guanghong, et al. Recent advances in multi-radar collaborative surveillance: Cognitive tracking and resource scheduling algorithms[J]. Journal of Radars, 2023, 12(3): 471–499. doi: 10.12000/JR23036.
[12]	ZHANG Peng, YAN Junkun, PU Wenqiang, et al. Collaborative strategy learning based transmit resource management scheme for multiple target tracking in multi-cooperative jamming environment[J]. IEEE Transactions on Signal Processing, 2026, 74: 2053–2068. doi: 10.1109/TSP.2026.3693184.
[13]	陈佳美, 孙慧雯, 李玉峰, 等. 基于双深度Q网络算法的无人机辅助密集网络资源优化策略[J]. 电子与信息学报, 2025, 47(8): 2621–2629. doi: 10.11999/JEIT250021. CHEN Jiamei, SUN Huiwen, LI Yufeng, et al. Double deep Q network algorithm-based unmanned aerial vehicle-assisted dense network resource optimization strategy[J]. Journal of Electronics & Information Technology, 2025, 47(8): 2621–2629. doi: 10.11999/JEIT250021.
[14]	周黎鸣, 余汐, 范明虎, 等. 基于双深度Q网络的多目标遥感产品生产任务调度算法[J]. 电子与信息学报, 2025, 47(8): 2819–2829. doi: 10.11999/JEIT250089. ZHOU Liming, YU Xi, FAN Minghu, et al. Multi-objective remote sensing product production task scheduling algorithm based on double deep Q-network[J]. Journal of Electronics & Information Technology, 2025, 47(8): 2819–2829. doi: 10.11999/JEIT250089.
[15]	李一兵, 孙柳晴, 戚昌龙. 基于改进秘书鸟算法的协同干扰资源分配方法[J]. 电子与信息学报, 2025, 47(5): 1494–1504. doi: 10.11999/JEIT240709. LI Yibing, SUN Liuqing, and QI Changlong. Collaborative interference resource allocation method based on improved secretary bird algorithm[J]. Journal of Electronics & Information Technology, 2025, 47(5): 1494–1504. doi: 10.11999/JEIT240709.