Labeled Multi-Bernoulli Sensor Management Strategy Based on Twin-Delayed Deep Deterministic Policy Gradient Learning Mechanism

ZHANG Xin-di; CHEN Hui; ZHANG Hong-yun; LIAN Feng; ZHANG Guang-hua; YIN Zhi-peng

doi:10.11999/JEIT260045

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 >

ZHANG Xin-di, CHEN Hui, ZHANG Hong-yun, LIAN Feng, ZHANG Guang-hua, YIN Zhi-peng. Labeled Multi-Bernoulli Sensor Management Strategy Based on Twin-Delayed Deep Deterministic Policy Gradient Learning Mechanism[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260045

Citation:

ZHANG Xin-di, CHEN Hui, ZHANG Hong-yun, LIAN Feng, ZHANG Guang-hua, YIN Zhi-peng. Labeled Multi-Bernoulli Sensor Management Strategy Based on Twin-Delayed Deep Deterministic Policy Gradient Learning Mechanism[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260045

Citation:

ZHANG Xin-di, CHEN Hui, ZHANG Hong-yun, LIAN Feng, ZHANG Guang-hua, YIN Zhi-peng. Labeled Multi-Bernoulli Sensor Management Strategy Based on Twin-Delayed Deep Deterministic Policy Gradient Learning Mechanism[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260045

PDF( 1794 KB)

Labeled Multi-Bernoulli Sensor Management Strategy Based on Twin-Delayed Deep Deterministic Policy Gradient Learning Mechanism

doi: 10.11999/JEIT260045 cstr: 32379.14.JEIT260045

1.
School of Automation and Electrical Engineering, Lanzhou University of Technology, Lanzhou 730050, China
2.
School of Automation Science and Engineering, Xi'an Jiaotong University, Xi’an 710049, China
3.
Hebei Xintian Kechuang New Energy Technology Co., Ltd., Zhangjiakou 075000, China

Funds: The National Natural Science Foundation of China (62163023, 61873116, 62366031, 62363023), the Gansu Provincial Science and Technology Plan Project of China (25ZYJA040, 25JRRA058), the 2024 Gansu Provincial Key Talent Project of China and the 2023 Gansu Provincial Special Fund for Military-Civilian Integration Development of China

Received Date: 2024-09-27
Accepted Date: 2026-06-30
Rev Recd Date: 2026-06-30

Available Online: 2026-07-13

Abstract

Abstract

Objective Multi-target tracking requires sensor management to adapt the observation process to clutter, missed detections, target-number variations, and changes in target motion. Conventional methods often search over a finite set of sensor actions, resulting in increasing computational cost and limited control resolution. Moreover, rewards formed by combining several single-target quantities may not adequately represent the joint multi-target posterior. A continuous-action sensor management method integrating the Twin-Delayed Deep Deterministic Policy Gradient algorithm with the Labeled Multi-Bernoulli filter is therefore developed to optimize mobile-sensor heading according to the multi-target belief state. Methods The LMB posterior, including target existence probabilities and state densities, is used to construct the belief state. At each filtering step, the mobile sensor selects a continuous heading angle that affects the sensor-target geometry, detection probability, and LMB update. Predicted target states and candidate headings are used to generate ideal measurements and obtain pseudo-updated LMB densities. The Cauchy-Schwarz divergence between the predicted and pseudo-updated densities is used to construct the information-gain reward. TD3 employs two critics, target policy smoothing, and delayed actor updates to reduce value-estimation bias. Random control, policy-gradient control, an information-driven discrete method, and DDPG-based continuous control are used for comparison. Results and Discussions DDPG-LMB and TD3-LMB produce more continuous steering changes than the discrete-action methods (Fig. 2). TD3-LMB obtains the highest or near-highest detection probabilities for most targets (Fig. 3) and gives relatively large Cauchy-Schwarz divergence values during most time steps, while random control remains relatively low (Fig. 4). TD3-LMB also achieves the lowest overall OSPA in the tested scenario, with DDPG-LMB generally outperforming the discrete baselines (Fig. 5). These results show that continuous heading control improves observation quality and overall LMB tracking performance. Conclusions A TD3-based continuous-action sensor management framework for the LMB filter is presented. Candidate heading actions are evaluated through pseudo-updated LMB densities and Cauchy-Schwarz divergence, linking action selection directly to the joint multi-target posterior. The simulation results show more continuous sensor motion, higher detection probabilities for most targets, larger information gain, and lower OSPA in the tested scenario. Future work will consider higher-dimensional actions and cooperative multi-sensor management.
- Multi-target tracking,
- Sensor management,
- Deep reinforcement learning,
- Label Multi-Bernoulli filter

FullText(HTML)

References(17)

References

[1]	LIU Zhihao, SHANG Yuanyuan, LI Timing, et al. Robust multi-drone multi-target tracking to resolve target occlusion: A benchmark[J]. IEEE Transactions on Multimedia, 2023, 25: 1462–1476. doi: 10.1109/TMM.2023.3234822.
[2]	HU Juan, ZUO Lei, VARSHNEY P K, et al. Resource allocation for distributed multitarget tracking in radar networks with missing data[J]. IEEE Transactions on Signal Processing, 2024, 72: 718–734. doi: 10.1109/TSP.2024.3352915.
[3]	COX P B and VAN ROSSUM W L. Split-aperture phased array radar resource management for tracking tasks[J]. IEEE Transactions on Aerospace and Electronic Systems, 2025, 61(3): 6476–6486. doi: 10.1109/TAES.2025.3531843.
[4]	HAYKIN S. Cognitive radar: A way of the future[J]. IEEE Signal Processing Magazine, 2006, 23(1): 30–40. doi: 10.1109/MSP.2006.1593335.
[5]	ZHANG Peng, YAN Junkun, PU Wenqiang, et al. Multi-dimensional resource management scheme for multiple target tracking under dynamic electromagnetic environment[J]. IEEE Transactions on Signal Processing, 2024, 72: 2377–2393. doi: 10.1109/TSP.2024.3390119.
[6]	CHARLISH A, HOFFMANN F, DEGEN C, et al. The development from adaptive to cognitive radar resource management[J]. IEEE Aerospace and Electronic Systems Magazine, 2020, 35(6): 8–19. doi: 10.1109/MAES.2019.2957847.
[7]	KURNIAWATI H. Partially observable Markov decision processes and robotics[J]. Annual Review of Control, Robotics, and Autonomous Systems, 2022, 5(1): 253–277. doi: 10.1146/annurev-control-042920-092451.
[8]	MAHLER R. Objective functions for Bayesian control-theoretic sensor management, II: MНC-like approximation[M]. BUTENKO S, MURPHEY R, and PARDALOS P M. Recent Developments in Cooperative Control and Optimization. Boston: Springer, 2004: 273–316. doi: 10.1007/978-1-4613-0219-3_16.
[9]	GOSTAR A K, HOSEINNEZHAD R, BAB-HADIASHAR A, et al. OSPA-based sensor control[C]. 2015 International Conference on Control, Automation and Information Sciences (ICCAIS), Changshu, China, 2015: 214–218. doi: 10.1109/ICCAIS.2015.7338664.
[10]	JONES G, GARCÍA-FERNÁNDEZ Á F, and WONG P W. GOSPA-driven Gaussian Bernoulli sensor management[C]. 2023 26th International Conference on Information Fusion (FUSION), Charleston, USA, 2023: 1–8. doi: 10.23919/FUSION52260.2023.10224220.
[11]	GOSTAR A K, HOSEINNEZHAD R, and BAB-HADIASHAR A. Multi-Bernoulli sensor control for multi-target tracking[C]. 2013 IEEE 8th International Conference on Intelligent Sensors, Sensor Networks and Information Processing, Melbourne, Australia, 2013: 312–317. doi: 10.1109/ISSNIP.2013.6529808.
[12]	RISTIC B, VO B N, and CLARK D. A note on the reward function for PHD filters with sensor control[J]. IEEE Transactions on Aerospace and Electronic Systems, 2011, 47(2): 1521–1529. doi: 10.1109/TAES.2011.5751278.
[13]	GOSTAR A K, HOSEINNEZHAD R, RATHNAYAKE T, et al. Constrained sensor control for labeled multi-Bernoulli filter using Cauchy-Schwarz divergence[J]. IEEE Signal Processing Letters, 2017, 24(9): 1313–1317. doi: 10.1109/LSP.2017.2723924.
[14]	HOFFMANN F, CHARLISH A, RITCHIE M, et al. Sensor path planning using reinforcement learning[C]. 2020 IEEE 23rd International Conference on Information Fusion (FUSION), Rustenburg, South Africa, 2020: 1–8. doi: 10.23919/FUSION45008.2020.9190242.
[15]	SHI Yuchun, JIU Bo, YAN Junkun, et al. Data-driven simultaneous multibeam power allocation: When multiple targets tracking meets deep reinforcement learning[J]. IEEE Systems Journal, 2021, 15(1): 1264–1274. doi: 10.1109/JSYST.2020.2984774.
[16]	LU Ziyang and GURSOY M C. Resource allocation for multi-target radar tracking via constrained deep reinforcement learning[J]. IEEE Transactions on Cognitive Communications and Networking, 2023, 9(6): 1677–1690. doi: 10.1109/TCCN.2023.3304634.
[17]	OBOREH-SNAPPS O, SHE Buxin, FAHAD S, et al. Virtual synchronous generator control using twin delayed deep deterministic policy gradient method[J]. IEEE Transactions on Energy Conversion, 2024, 39(1): 214–228. doi: 10.1109/TEC.2023.3309955.