Trusted Geographic Routing Protocol Based on Deep Reinforcement Learning for Unmanned Aerial Vehicle Network

ZHANG Yanan; QIU Hongbing

doi:10.11999/JEIT220649

Volume 44 Issue 12

Dec. 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 > 44(12): 4211-4217

ZHANG Yanan, QIU Hongbing. Trusted Geographic Routing Protocol Based on Deep Reinforcement Learning for Unmanned Aerial Vehicle Network[J]. Journal of Electronics & Information Technology, 2022, 44(12): 4211-4217. doi: 10.11999/JEIT220649

Citation:

ZHANG Yanan, QIU Hongbing. Trusted Geographic Routing Protocol Based on Deep Reinforcement Learning for Unmanned Aerial Vehicle Network[J]. Journal of Electronics & Information Technology, 2022, 44(12): 4211-4217. doi: 10.11999/JEIT220649

Citation:

PDF( 2940 KB)

Trusted Geographic Routing Protocol Based on Deep Reinforcement Learning for Unmanned Aerial Vehicle Network

doi: 10.11999/JEIT220649

ZHANG Yanan^,,
QIU Hongbing

1.
School of Information and Communications, Guilin University of Electronic Technology, Guilin 541004, China
2.
Guangxi Key Laboratory of Wireless Wideband Communication and Signal Processing, Guilin 541004, China

Funds: The Natural Science Foundation of Guangxi (2022GXNSFDA035070)

Received Date: 2022-05-19
Rev Recd Date: 2022-07-01

Available Online: 2022-07-05

Publish Date: 2022-12-16

Abstract

Abstract

Considering the problems of high mobility and abnormal nodes in Unmanned Aerial Vehicle (UAV) communication, a Deep reinforcement learning based Trusted Geographic Routing protocol (DTGR) is proposed. A trusted third party is introduced to provide the trust of nodes. The difference between theoretical delay and real delay, and packet loss ratio are used as evaluation factors of trust degree. Routing selection is modeled as the Markov Decision Process (MDP). The state are constructed based on the neighbor nodes’ geographic location, the trust degree and the topology information. Then the routing decision can be output through the Deep Q Network(DQN). The action-value is adjusted by combining trust in reward function, to guide nodes to select the optimal next-hop. The simulation results show that DTGR has a lower average end-to-end delay and higher packet delivery ratio compared with existing schemes in UAV Ad hoc NETwork (UANET) with abnormal nodes. Besides, DTGR can effectively implement route selection and ensure network performance when the number or proportion of abnormal nodes changes.
- Unmanned Aerial Vehicle (UAV) routing protocol,
- Trust degree,
- Markov Decision Process (MDP),
- Deep reinforcement learning

FullText(HTML)

References(16)

References

[1]	BUSHNAQ O M, CHAABAN A, and AL-NAFFOURI T Y. The role of UAV-IoT networks in future wildfire detection[J]. IEEE Internet of Things Journal, 2021, 8(23): 16984–16999. doi: 10.1109/JIOT.2021.3077593
[2]	FENG Wanmei, TANG Jie, YU Yu, et al. UAV-enabled SWIPT in IoT networks for emergency communications[J]. IEEE Wireless Communications, 2020, 27(5): 140–147. doi: 10.1109/MWC.001.1900656
[3]	BOURSIANIS A D, PAPADOPOULOU M S, DIAMANTOULAKIS P, et al. Internet of things (IoT) and agricultural unmanned aerial vehicles (UAVs) in smart farming: A comprehensive review[J]. Internet of Things, 2022, 18: 100187. doi: 10.1016/j.iot.2020.100187
[4]	KHAWAJA W, OZDEMIR O, and GUVENC I. UAV air-to-ground channel characterization for mmWave systems[C]. IEEE 86th Vehicular Technology Conference (VTC-Fall), Toronto, Canada, 2017: 1–5.
[5]	CHEN I R, BAO Fenye, CHANG M J, et al. Dynamic trust management for delay tolerant networks and its application to secure routing[J]. IEEE Transactions on Parallel and Distributed Systems, 2014, 25(5): 1200–1210. doi: 10.1109/TPDS.2013.116
[6]	CHO J H, SWAMI A, and CHEN I R. A survey on trust management for mobile ad hoc networks[J]. IEEE Communications Surveys & Tutorials, 2011, 13(4): 562–583. doi: 10.1109/SURV.2011.092110.00088
[7]	HYLAND M T, MULLINS B E, BALDWIN R O, et al. Simulation-based performance evaluation of mobile ad hoc routing protocols in a swarm of unmanned aerial vehicles[C]. 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW'07), Niagara Falls, Canada, 2007: 249–256.
[8]	KARP B and KUNG H T. GPSR: Greedy perimeter stateless routing for wireless networks[C]. The 6th Annual International Conference on Mobile Computing and Networking, Boston, USA, 2000: 243–254.
[9]	JUNG W S, YIM J, and KO Y B. QGeo: Q-learning-based geographic ad hoc routing protocol for unmanned robotic networks[J]. IEEE Communications Letters, 2017, 21(10): 2258–2261. doi: 10.1109/LCOMM.2017.2656879
[10]	LYU Niqi, SONG Guanghua, YANG Bowei, et al. QNGPSR: A Q-network enhanced geographic ad-hoc routing protocol based on GPSR[C]. 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall), Chicago, USA, 2018: 1–6.
[11]	LIU Jianmin, WANG Qi, HE Chentao, et al. QMR: Q-learning based multi-objective optimization routing protocol for flying ad hoc networks[J]. Computer Communications, 2020, 150: 304–316. doi: 10.1016/j.comcom.2019.11.011
[12]	ZHU Haojin, DU Suguo, GAO Zhaoyu, et al. A probabilistic misbehavior detection scheme toward efficient trust establishment in delay-tolerant networks[J]. IEEE Transactions on Parallel and Distributed Systems, 2014, 25(1): 22–32. doi: 10.1109/TPDS.2013.36
[13]	CHO J H and CHEN I R. PROVEST: Provenance-based trust model for delay tolerant networks[J]. IEEE Transactions on Dependable and Secure Computing, 2018, 15(1): 151–165. doi: 10.1109/TDSC.2016.2530705
[14]	GE Chunpeng, ZHOU Lu, HANCKE G P, et al. A provenance-aware distributed trust model for resilient unmanned aerial vehicle networks[J]. IEEE Internet of Things Journal, 2021, 8(16): 12481–12489. doi: 10.1109/JIOT.2020.3014947
[15]	DJEDJIG N, TANDJAOUI D, MEDJEK F, et al. Trust-aware and cooperative routing protocol for IoT security[J]. Journal of Information Security and Applications, 2020, 52: 102467. doi: 10.1016/j.jisa.2020.102467
[16]	MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529–533. doi: 10.1038/nature14236