Fourier Personalized Federated Learning Mechanism Based on Spectral Clustering

JIN Tong; CHEN Siguang

doi:10.11999/JEIT220529

Volume 45 Issue 6

Jun. 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2023 > 45(6): 1981-1989

JIN Tong, CHEN Siguang. Fourier Personalized Federated Learning Mechanism Based on Spectral Clustering[J]. Journal of Electronics & Information Technology, 2023, 45(6): 1981-1989. doi: 10.11999/JEIT220529

Citation:

JIN Tong, CHEN Siguang. Fourier Personalized Federated Learning Mechanism Based on Spectral Clustering[J]. Journal of Electronics & Information Technology, 2023, 45(6): 1981-1989. doi: 10.11999/JEIT220529

Citation:

PDF( 3760 KB)

Fourier Personalized Federated Learning Mechanism Based on Spectral Clustering

doi: 10.11999/JEIT220529

JIN Tong,
CHEN Siguang^,

School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing 210003, China

Funds: The National Natural Science Foundation of China (61971235), China Postdoctoral Science Foundation (2018M630590), 333 High-level Talents Training Project of Jiangsu Province, Jiangsu Planned Projects for Postdoctoral Research Funds (2021K501C), 1311 Talents Plan of NJUPT, The Jiangsu Postgraduate Scientific Research Innovation Plan (KYCX22_1029)

Received Date: 2022-04-27
Accepted Date: 2022-12-20
Rev Recd Date: 2022-12-07

Available Online: 2022-12-23

Publish Date: 2023-06-10

Abstract

Abstract

To relieve the negative impacts caused by non-Independent and Identically Distributed (non-IID) data across different clients in federated learning, a spectral clustering-based Fourier personalized federated learning mechanism is proposed to overcome the performance drops from data heterogeneity. Specifically, a cloud-edge-end collaborative personalized federated learning model for image recognition is constructed, and in order to make full use of the knowledge learned by similar clients, the clients are divided into multiple clusters by spectral clustering under cloud-edge collaboration. Next, a local federated learning method based on edge-end collaboration is proposed, in which an agent model is used to perform the process of restoring and re-updating the personalized local model at the clients to restore the local knowledge loss during aggregation. Furthermore, a cloud-edge collaborative Fourier personalized federated learning method is proposed to adapt the global model to each distributed client. In this method, the cloud server converts the local model parameters to the frequency domain space for aggregation through Fourier transform, and customizes high-quality personalized local model for each edge node. Finally, the experimental results demonstrate that the proposed algorithm obtains competitive convergence speed compared with existing representative works and the accuracy is 3%～13% higher.
- Edge computing,
- Federated learning,
- Spectral clustering,
- Fourier transform

FullText(HTML)

References(19)

References

[1]	MOHAMMADI M, AL-FUQAHA A, GUIZANI M, et al. Semisupervised deep reinforcement learning in support of IoT and smart city services[J]. IEEE Internet of Things Journal, 2018, 5(2): 624–635. doi: 10.1109/jiot.2017.2712560
[2]	LI Chengxi, LI Gang, and VARSHNEY P K. Federated learning with soft clustering[J]. IEEE Internet of Things Journal, 2022, 9(10): 7773–7782. doi: 10.1109/JIOT.2021.3113927
[3]	MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]. The 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA, 2017: 1273–1282.
[4]	FEKI I, AMMAR S, KESSENTINI Y, et al. Federated learning for COVID-19 screening from chest X-ray images[J]. Applied Soft Computing, 2021, 106: 107330. doi: 10.1016/j.asoc.2021.107330
[5]	SILVA S, GUTMAN B A, ROMERO E, et al. Federated learning in distributed medical databases: Meta-analysis of large-scale subcortical brain data[C]. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI), Venice, Italy, 2019: 270–274.
[6]	LIU Yang, HUANG Anbu, LUO Yun, et al. FedVision: An online visual object detection platform powered by federated learning[C]. The AAAI Conference on Artificial Intelligence, New York, USA, 2020: 13172–13179.
[7]	ZHAO Zhongyuan, FENG Chenyuan, HONG Wei, et al. Federated learning with non-IID data in wireless networks[J]. IEEE Transactions on Wireless Communications, 2022, 21(3): 1927–1942. doi: 10.1109/TWC.2021.3108197
[8]	KULKARNI V, KULKARNI M, and PANT A. Survey of personalization techniques for federated learning[C]. The 4th World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), London, UK, 2020: 794–797.
[9]	YAN Zengqiang, WICAKSANA J, WANG Zhiwei, et al. Variation-aware federated learning with multi-source decentralized medical image data[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(7): 2615–2628. doi: 10.1109/JBHI.2020.3040015
[10]	HUANG Li, SHEA A L, QIAN Huining, et al. Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records[J]. Journal of Biomedical Informatics, 2019, 99: 103291. doi: 10.1016/j.jbi.2019.103291
[11]	SATTLER F, MÜLLER K R, and SAMEK W. Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(8): 3710–3722. doi: 10.1109/TNNLS.2020.3015958
[12]	BRIGGS C, FAN Zhong, and ANDRAS P. Federated learning with hierarchical clustering of local updates to improve training on non-IID data[C]. 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK, 2020: 1–9.
[13]	EK S, PORTET F, LALANDA P, et al. Artifact: A federated learning aggregation algorithm for pervasive computing: Evaluation and comparison[C]. 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), Kassel, Germany, 2021: 448–449.
[14]	LI Xiaoxiao, JIANG Meirui, ZHANG Xiaofei, et al. FedBN: Federated learning on non-IID features via local batch normalization[C]. 9th International Conference on Learning Representations (ICLR), Vienna, Austria, 2021: 1–27.
[15]	CHEN Hongyou and CHAO Weilun. FedBE: Making Bayesian model ensemble applicable to federated learning[C]. 9th International Conference on Learning Representations (ICLR), Vienna, Austria, 2020: 1–21.
[16]	YE Dongdong, YU Rong, PAN Miao, et al. Federated learning in vehicular edge computing: A selective model aggregation approach[J]. IEEE Access, 2020, 8: 23920–23935. doi: 10.1109/ACCESS.2020.2968399
[17]	HUANG Yutao, CHU Lingyang, ZHOU Zirui, et al. Personalized cross-silo federated learning on non-IID data[J/OL]. The AAAI Conference on Artificial Intelligence, 2021: 7865–7873.
[18]	ZHANG M, SAPRA K, FIDLER S, et al. Personalized federated learning with first order model optimization[C]. 9th International Conference on Learning Representations (ICLR), Vienna, Austria, 2020: 1–17.
[19]	CHEN Zhen, ZHU Meilu, YANG Chen, et al. Personalized retrogress-resilient framework for real-world medical federated learning[C]. The 24th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Strasbourg, France, 2021: 347–356.