高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于谱聚类的傅里叶个性化联邦学习研究

金彤 陈思光

金彤, 陈思光. 基于谱聚类的傅里叶个性化联邦学习研究[J]. 电子与信息学报, 2023, 45(6): 1981-1989. doi: 10.11999/JEIT220529
引用本文: 金彤, 陈思光. 基于谱聚类的傅里叶个性化联邦学习研究[J]. 电子与信息学报, 2023, 45(6): 1981-1989. doi: 10.11999/JEIT220529
JIN Tong, CHEN Siguang. Fourier Personalized Federated Learning Mechanism Based on Spectral Clustering[J]. Journal of Electronics & Information Technology, 2023, 45(6): 1981-1989. doi: 10.11999/JEIT220529
Citation: JIN Tong, CHEN Siguang. Fourier Personalized Federated Learning Mechanism Based on Spectral Clustering[J]. Journal of Electronics & Information Technology, 2023, 45(6): 1981-1989. doi: 10.11999/JEIT220529

基于谱聚类的傅里叶个性化联邦学习研究

doi: 10.11999/JEIT220529
基金项目: 国家自然科学基金(61971235),中国博士后科学基金(2018M630590),江苏省“333高层次人才培养工程”,江苏博士后科研资助计划(2021K501C),南邮“1311”人才计划和江苏研究生科研创新计划(KYCX22_1029)
详细信息
    作者简介:

    金彤:女,博士生,研究方向为联邦学习与边缘智能等

    陈思光:男,博士,教授,研究方向为边缘智能、智慧物联网等

    通讯作者:

    陈思光 sgchen@njupt.edu.cn

  • 中图分类号: TN919.2; TP393

Fourier Personalized Federated Learning Mechanism Based on Spectral Clustering

Funds: The National Natural Science Foundation of China (61971235), China Postdoctoral Science Foundation (2018M630590), 333 High-level Talents Training Project of Jiangsu Province, Jiangsu Planned Projects for Postdoctoral Research Funds (2021K501C), 1311 Talents Plan of NJUPT, The Jiangsu Postgraduate Scientific Research Innovation Plan (KYCX22_1029)
  • 摘要: 为了缓解联邦学习中跨不同用户终端数据非独立同分布(non-IID)引起的负面影响,该文提出一种基于谱聚类的傅里叶个性化联邦学习算法。具体地,构建一个面向图像分类识别的云边端协同个性化联邦学习模型,提出在云端协同下通过谱聚类将用户终端划分为多个聚类域,以充分利用相似用户终端学到的知识提升模型性能。其次,设计边端协同的局部联邦学习方法,通过代理模型在用户终端对个性化局部模型执行恢复与再更新的操作,可有效恢复聚合过程中丢失的本地知识。进一步地,设计云边协同的傅里叶个性化联邦学习方法,即云服务器通过傅里叶变换将局部模型参数转换到频域空间上进行聚合,为每个边缘节点定制高质量的个性化局部模型,可使全局模型更适用于各个分布式用户终端。最后,实验结果表明,与现有相关算法相比,所提算法收敛速度更快,准确率提高了3%~13%。
  • 图  1  云边端协同个性化联邦学习模型

    图  2  个性化联邦学习框架

    图  3  收敛性分析

    图  4  不同模型准确率与聚类域个数关系对比

    图  5  不同模型准确率与用户终端个数关系对比

    图  6  不同模型平均更新范数与通信轮数之间关系对比

    算法1 基于谱聚类的傅里叶个性化联邦学习算法
     输入:${D_i}(1 \le i \le N)$, ${D_p} $, $f $和通信轮数V.
     输出:${f_i}(1 \le i \le N)$.
     1. BEGIN
     2. 用户终端i从云服务器下载初始训练模型$f $到本地;
     3. 用户终端i利用本地私有数据${D_i} $训练更新本地模型${f_i} $;
     4. 用户终端$i $将${D_p} $的预测结果矩阵${{\mathbf{P}}_i} = ({p_{i1}},{p_{i2}}, \cdots ,{p_{iR}}) $上传至
      云服务器;
     5. 云服务器利用式(1)为用户终端计算相似矩阵${{\mathbf{S}}_{N \times N}} $;
     6. 云服务器利用式(2)—式(8),通过谱聚类将$N $个用户终端聚类
      成$K $个互不相交的聚类域$\displaystyle\bigcup\limits_{k = 1}^K { {\text{cluste} }{ {\text{r} }_k} } = \left\{ {1,2, \cdots ,N} \right\}$;
     7. 设置迭代次数$v = 1 $;
     8. 本轮迭代开始:
     9. 用户终端$ i \in {\text{cluste}}{{\text{r}}_k} $将本地模型参数${f_i} $上传给同一边缘节点
      ${B_k} $,聚合为局部模型${\theta _k} $;
     10. 边缘节点将局部模型${\theta _k} $上传给云服务器;
     11. 云服务器利用式 (13)—式(17) ,采用傅里叶聚合为每个边缘
      节点${B_k} $定制相应的个性化局部模型${\theta _k} $;
     12. 用户终端$i $下载个性化局部模型${\theta _k} $到本地代理模型${d_i} $,利用
      式(10)—式(12)恢复聚合过程中丢失的知识,将聚合到的知识
      传输到本地模型;
     13. $v = v + 1 $;
     14. 若$v < V $,重复步骤7—步骤13,否则跳出迭代;
     15. 得到个性化本地模型参数${f_i}(1 \le i \le N)$.
     16. END
    下载: 导出CSV
  • [1] MOHAMMADI M, AL-FUQAHA A, GUIZANI M, et al. Semisupervised deep reinforcement learning in support of IoT and smart city services[J]. IEEE Internet of Things Journal, 2018, 5(2): 624–635. doi: 10.1109/jiot.2017.2712560
    [2] LI Chengxi, LI Gang, and VARSHNEY P K. Federated learning with soft clustering[J]. IEEE Internet of Things Journal, 2022, 9(10): 7773–7782. doi: 10.1109/JIOT.2021.3113927
    [3] MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]. The 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA, 2017: 1273–1282.
    [4] FEKI I, AMMAR S, KESSENTINI Y, et al. Federated learning for COVID-19 screening from chest X-ray images[J]. Applied Soft Computing, 2021, 106: 107330. doi: 10.1016/j.asoc.2021.107330
    [5] SILVA S, GUTMAN B A, ROMERO E, et al. Federated learning in distributed medical databases: Meta-analysis of large-scale subcortical brain data[C]. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI), Venice, Italy, 2019: 270–274.
    [6] LIU Yang, HUANG Anbu, LUO Yun, et al. FedVision: An online visual object detection platform powered by federated learning[C]. The AAAI Conference on Artificial Intelligence, New York, USA, 2020: 13172–13179.
    [7] ZHAO Zhongyuan, FENG Chenyuan, HONG Wei, et al. Federated learning with non-IID data in wireless networks[J]. IEEE Transactions on Wireless Communications, 2022, 21(3): 1927–1942. doi: 10.1109/TWC.2021.3108197
    [8] KULKARNI V, KULKARNI M, and PANT A. Survey of personalization techniques for federated learning[C]. The 4th World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), London, UK, 2020: 794–797.
    [9] YAN Zengqiang, WICAKSANA J, WANG Zhiwei, et al. Variation-aware federated learning with multi-source decentralized medical image data[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(7): 2615–2628. doi: 10.1109/JBHI.2020.3040015
    [10] HUANG Li, SHEA A L, QIAN Huining, et al. Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records[J]. Journal of Biomedical Informatics, 2019, 99: 103291. doi: 10.1016/j.jbi.2019.103291
    [11] SATTLER F, MÜLLER K R, and SAMEK W. Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(8): 3710–3722. doi: 10.1109/TNNLS.2020.3015958
    [12] BRIGGS C, FAN Zhong, and ANDRAS P. Federated learning with hierarchical clustering of local updates to improve training on non-IID data[C]. 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK, 2020: 1–9.
    [13] EK S, PORTET F, LALANDA P, et al. Artifact: A federated learning aggregation algorithm for pervasive computing: Evaluation and comparison[C]. 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), Kassel, Germany, 2021: 448–449.
    [14] LI Xiaoxiao, JIANG Meirui, ZHANG Xiaofei, et al. FedBN: Federated learning on non-IID features via local batch normalization[C]. 9th International Conference on Learning Representations (ICLR), Vienna, Austria, 2021: 1–27.
    [15] CHEN Hongyou and CHAO Weilun. FedBE: Making Bayesian model ensemble applicable to federated learning[C]. 9th International Conference on Learning Representations (ICLR), Vienna, Austria, 2020: 1–21.
    [16] YE Dongdong, YU Rong, PAN Miao, et al. Federated learning in vehicular edge computing: A selective model aggregation approach[J]. IEEE Access, 2020, 8: 23920–23935. doi: 10.1109/ACCESS.2020.2968399
    [17] HUANG Yutao, CHU Lingyang, ZHOU Zirui, et al. Personalized cross-silo federated learning on non-IID data[J/OL]. The AAAI Conference on Artificial Intelligence, 2021: 7865–7873.
    [18] ZHANG M, SAPRA K, FIDLER S, et al. Personalized federated learning with first order model optimization[C]. 9th International Conference on Learning Representations (ICLR), Vienna, Austria, 2020: 1–17.
    [19] CHEN Zhen, ZHU Meilu, YANG Chen, et al. Personalized retrogress-resilient framework for real-world medical federated learning[C]. The 24th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Strasbourg, France, 2021: 347–356.
  • 加载中
图(6) / 表(1)
计量
  • 文章访问数:  439
  • HTML全文浏览量:  289
  • PDF下载量:  114
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-04-27
  • 修回日期:  2022-12-07
  • 录用日期:  2022-12-20
  • 网络出版日期:  2022-12-23
  • 刊出日期:  2023-06-10

目录

    /

    返回文章
    返回