高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

双因子更新的车联网双层异步联邦学习研究

王力立 吴守林 杨妮 黄成

王力立, 吴守林, 杨妮, 黄成. 双因子更新的车联网双层异步联邦学习研究[J]. 电子与信息学报. doi: 10.11999/JEIT230918
引用本文: 王力立, 吴守林, 杨妮, 黄成. 双因子更新的车联网双层异步联邦学习研究[J]. 电子与信息学报. doi: 10.11999/JEIT230918
WANG Lili, WU Shoulin, YANG Ni, HUANG Cheng. A Study of Two-layer Asynchronous Federated Learning with Two-factor Updating for Vehicular Networking[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT230918
Citation: WANG Lili, WU Shoulin, YANG Ni, HUANG Cheng. A Study of Two-layer Asynchronous Federated Learning with Two-factor Updating for Vehicular Networking[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT230918

双因子更新的车联网双层异步联邦学习研究

doi: 10.11999/JEIT230918
详细信息
    作者简介:

    王力立:女,副教授,研究方向为深度强化学习及无线传感网络

    吴守林:男,硕士生,研究方向为车联网中的联邦学习、任务卸载

    杨妮:女,硕士生,研究方向为联邦学习、知识交易

    黄成:男,讲 师,研究方向为机器学习及检测技术

    通讯作者:

    王力立 liliwang@njust.edu.cn

  • 中图分类号: TN929.5

A Study of Two-layer Asynchronous Federated Learning with Two-factor Updating for Vehicular Networking

  • 摘要: 针对车联网(IoV)中节点资源异构、拓扑结构动态变化等特点,该文建立了一个双因子更新的双层异步联邦学习(TTAFL)框架。考虑到模型版本差和车辆参与联邦学习(FL)次数对局部模型更新的影响,提出基于陈旧因子和贡献因子的模型更新方案。同时,为了避免训练过程中,车辆移动带来路侧单元切换的问题,给出考虑驻留时间的节点选择方案。最后,为了减少精度损失与系统能耗,利用强化学习方法优化联邦学习的本地迭代次数与路侧单元局部模型更新次数。仿真结果表明,所提算法有效提高了联邦学习的训练效率和训练精度,降低了系统能耗。
  • 图  1  车联网中的双层异步联邦学习框架

    图  2  DDQN算法流程图

    图  3  强化学习的训练奖励

    图  4  强化学习的训练损失

    图  5  不同车辆总数及路侧单元数对精度的影响

    图  6  各RSU局部模型精度

    图  7  训练精度对比

    图  8  基于不同选择策略的能耗与时延分析

     算法1 基于DDQN的联邦学习参数优化算法
     输入:迭代轮数$T$,动作集$A$,衰减因子$\gamma $,探索率$\varepsilon $,探索衰
     减$ {\text{decay}} $,当前$Q$网络$Q$,Target $Q$网络$Q'$,批量梯度下降的样
     本数$h$,Target $Q$网络参数更新频率$H$,神经网络开始训练时系
     统当前轮次为$ {P_1} $,神经网络训练的频率为$ {P_2} $
     输出最优动作$ a = (\theta ,\kappa ) $
     (1) 初始化当前$Q$网络与Target $Q$网络,将经验回放池
       Memory清空。
     (2)  for epoch=1 to $T$ do
     (3) 初始化双层异步联邦学习的状态s
     (4) 将$s$输入$Q$网络,得到$Q$网络所有动作对应的$Q$值输出,并
       利用$ \varepsilon $-贪婪策略选出动作$ a = (\theta ,\kappa ) $
     (5) 根据动作更新联邦学习参数$ \theta $和$ \kappa $并进行训练,得到新的状态
       $ {s_{t + 1}} $和奖励$ {r_t} $
     (6) 将$ \{ {s_t},{a_t}.{r_t},{s_{t + 1}}\} $存入经验回放池
     (7) 令$ {s_t} = {s_{t + 1}} $
     (8)  if $ {\text{epoch}} \gt {P_1}{\text{ }}\& \& {\text{ epoch }}\% {\text{ }}{P_2} = = 0 $
     (9) 从经验回放池中采样$h$个样本,并计算Target $Q$的值$ {y_{t + 1}} $
     (10) 将误差$ {\text{loss}} = \dfrac{1}{h}\displaystyle\sum\limits_{i = 1}^h {{{({y_{t + 1}} - {Q^{{\text{Target}}}}({s_{t + 1}},{a_{t + 1}}))}^2}} $反向传
       播以更新$Q$网络参数
     (11) if $ {\text{epoch }}\% {\text{ }}H = = 0 $
     (12) 将$Q$网络参数复制给Target $Q$网络
     (13) end if
     (14) end if
     (15) $ {\text{epoch = epoch}} + 1 $
     (16) $ \varepsilon = \varepsilon \cdot {\text{decay}} $
     (17) end for
    下载: 导出CSV
  • [1] ZHOU Xiaokang, LIANG Wei, SHE Jinhua, et al. Two-layer federated learning with heterogeneous model aggregation for 6G supported internet of vehicles[J]. IEEE Transactions on Vehicular Technology, 2021, 70(6): 5308–5317. doi: 10.1109/TVT.2021.3077893
    [2] XU Chenhao, QU Youyang, LUAN T H, et al. An efficient and reliable asynchronous federated learning scheme for smart public transportation[J]. IEEE Transactions on Vehicular Technology, 2023, 72(5): 6584–6598. doi: 10.1109/TVT.2022.3232603
    [3] MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]. The 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA, 2017: 1273–1282.
    [4] ABDULRAHMAN S, TOUT H, OULD-SLIMANE H, et al. A survey on federated learning: The journey from centralized to distributed on-site learning and beyond[J]. IEEE Internet of Things Journal, 2021, 8(7): 5476–5497. doi: 10.1109/JIOT.2020.3030072
    [5] ZHU Hangyu, ZHANG Haoyu, and JIN Yaochu. From federated learning to federated neural architecture search: A survey[J]. Complex & Intelligent Systems, 2021, 7(2): 639–657. doi: 10.1007/s40747-020-00247-z
    [6] SUN Feng, ZHANG Zhenjiang, ZEADALLY S, et al. Edge computing-enabled internet of vehicles: Towards federated learning empowered scheduling[J]. IEEE Transactions on Vehicular Technology, 2022, 71(9): 10088–10103. doi: 10.1109/TVT.2022.3182782
    [7] LIM W Y B, LUONG N C, HOANG D T, et al. Federated learning in mobile edge networks: A comprehensive survey[J]. IEEE Communications Surveys & Tutorials, 2020, 22(3): 2031–2063. doi: 10.1109/COMST.2020.2986024
    [8] YANG Zhigang, ZHANG Xuhua, WU Dapeng, et al. Efficient asynchronous federated learning research in the internet of vehicles[J]. IEEE Internet of Things Journal, 2023, 10(9): 7737–7748. doi: 10.1109/JIOT.2022.3230412
    [9] CHAI Haoye, LENG Supeng, CHEN Yijin, et al. A hierarchical blockchain-enabled federated learning algorithm for knowledge sharing in internet of vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(7): 3975–3986. doi: 10.1109/TITS.2020.3002712
    [10] SAPUTRA Y M, HOANG D T, NGUYEN D N, et al. Dynamic federated learning-based economic framework for internet-of-vehicles[J]. IEEE Transactions on Mobile Computing, 2023, 22(4): 2100–2115. doi: 10.1109/TMC.2021.3122436
    [11] WANG Yuwei and KANTARCI B. A novel reputation-aware client selection scheme for federated learning within mobile environments[C]. 2020 IEEE 25th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks, Pisa, Italy, 2020: 1–6.
    [12] XIAO Huizi, ZHAO Jun, PEI Qingqi, et al. Vehicle selection and resource optimization for federated learning in vehicular edge computing[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(8): 11073–11087. doi: 10.1109/TITS.2021.3099597
    [13] 贺文晨, 郭少勇, 邱雪松, 等. 基于DRL的联邦学习节点选择方法[J]. 通信学报, 2021, 42(6): 62–71. doi: 10.11959/j.issn.1000-436x.2021111

    HE Wenchen, GUO Shaoyong, QIU Xuesong, et al. Node selection method in federated learning based on deep reinforcement learning[J]. Journal on Communications, 2021, 42(6): 62–71. doi: 10.11959/j.issn.1000-436x.2021111
    [14] YANG Peng, YAN Mengjiao, CUI Yaping, et al. FedDD: Federated double distillation in IoV[C]. 2022 IEEE 96th Vehicular Technology Conference, London, United Kingdom, 2022: 1–5.
    [15] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278–2324. doi: 10.1109/5.726791
  • 加载中
图(8) / 表(1)
计量
  • 文章访问数:  48
  • HTML全文浏览量:  27
  • PDF下载量:  11
  • 被引次数: 0
出版历程
  • 收稿日期:  2023-08-23
  • 修回日期:  2024-01-11
  • 网络出版日期:  2024-01-15

目录

    /

    返回文章
    返回