A Secure Gradient Aggregation Scheme based on Local Differential Privacy in Asynchronous Horizontal Federated Learning

WEI Lifei; ZHANG Wuji; ZHANG Lei; HU Xuehui; WANG Xuan

doi:10.11999/JEIT230923

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 >

WEI Lifei, ZHANG Wuji, ZHANG Lei, HU Xuehui, WANG Xuan. A Secure Gradient Aggregation Scheme based on Local Differential Privacy in Asynchronous Horizontal Federated Learning[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT230923

Citation:

WEI Lifei, ZHANG Wuji, ZHANG Lei, HU Xuehui, WANG Xuan. A Secure Gradient Aggregation Scheme based on Local Differential Privacy in Asynchronous Horizontal Federated Learning[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT230923

Citation:

WEI Lifei, ZHANG Wuji, ZHANG Lei, HU Xuehui, WANG Xuan. A Secure Gradient Aggregation Scheme based on Local Differential Privacy in Asynchronous Horizontal Federated Learning[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT230923

PDF( 1673 KB)

A Secure Gradient Aggregation Scheme based on Local Differential Privacy in Asynchronous Horizontal Federated Learning

doi: 10.11999/JEIT230923

WEI Lifei^{1, 2},
ZHANG Wuji¹,
ZHANG Lei^{1
,
,},
HU Xuehui³,
WANG Xuan⁴

1.
College of Information Technology, Shanghai Ocean University, Shanghai 201306, China
2.
College of Information Engineering, Shanghai Maritime University, Shanghai 201306, China
3.
Shanghai Tongtai Information Technology Co., Ltd., Shanghai 200235, China
4.
Engineering University of PAP, Xi’an Shaanxi 710086, China

Funds: The National Natural Science Foundation of China (61972241, 62172436), Natural Science Foundation of Shanghai (22ZR1427100), Natural Science Foundation of Shaanxi Province (2023-JC-YB-584), Soft Science Project of Shanghai (23692106700)

Received Date: 2023-08-28
Rev Recd Date: 2023-12-21

Available Online: 2023-12-26

Abstract

Abstract

Federated learning is an emerging distributed machine learning framework that effectively solves the problems of data silos and privacy leakage in traditional machine learning by performing joint modeling training without leaving the user’s private data out of the domain. However, federated learning suffers from the problem of training-lagged clients dragging down the global training speed. Related research has proposed asynchronous federated learning, which allows the users to upload to the server and participate in the aggregation task as soon as they finish updating their models locally, without waiting for the other users. However, asynchronous federated learning also suffers from the inability to recognize malicious models uploaded by malicious users and the problem of leaking user’s privacy. To address these issues, a privacy-preserving Secure Aggregation scheme for asynchronous Federated Learning(SAFL) is designed. The users add perturbations to locally trained models and upload the perturbed models to the server. The server detects and rejects the malicious users through a poisoning detection algorithm to achieve Secure Aggregation(SA). Finally, theoretical analysis and experiments show that in the scenario of asynchronous federated learning, the proposed scheme can effectively detect malicious users while protecting the privacy of users’ local models and reducing the risk of privacy leakage. The proposed scheme has also a significant improvement in the accuracy of the model compared with other schemes.
- Secure Aggregation (SA),
- Local differential privacy,
- Privacy preserving,
- Malicious poisoning attack,
- Asynchronous federated learning

FullText(HTML)

References(26)

References

[1]	MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA, 2017: 1273–82.
[2]	YANG Qiang, LIU Yang, CHEN Tianjian, et al. Federated machine learning: Concept and applications[J]. ACM Transactions on Intelligent Systems and Technology, 2019, 10(2): 12. doi: 10.1145/3298981
[3]	BONAWITZ K, EICHNER H, GRIESKAMP W, et al. Towards federated learning at scale: System design[C]. Proceedings of Machine Learning and Systems, Stanford, USA, 2019: 374–88.
[4]	XIE Cong, KOYEJO S, and GUPTA I. Asynchronous federated optimization[EB/OL]. https://arxiv.org/abs/1903.03934, 2019.
[5]	LIU Jianchun, XU Hongli, WANG Lun, et al. Adaptive asynchronous federated learning in resource-constrained edge computing[J]. IEEE Transactions on Mobile Computing, 2023, 22(2): 674–690. doi: 10.1109/TMC.2021.3096846
[6]	HUBA D, NGUYEN J, MALIK K, et al. Papaya: Practical, private, and scalable federated learning[C]. Proceedings of Machine Learning and Systems, Santa Clara, USA, 2022: 814–32.
[7]	BAGDASARYAN E, VEIT A, HUA Yiqing, et al. How to backdoor federated learning[C]. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, Palermo, Italy, 2020: 2938–2948.
[8]	高莹, 陈晓峰, 张一余, 等. 联邦学习系统攻击与防御技术研究综述[J]. 计算机学报, 2023, 46(9): 1781–1805. doi: 10.11897/SP.J.1016.2023.01781 GAO Ying, CHEN Xiaofeng, ZHANG Yiyu, et al. A survey of attack and defense techniques for federated learning systems[J]. Chinese Journal of Computers, 2023, 46(9): 1781–1805. doi: 10.11897/SP.J.1016.2023.01781
[9]	汤凌韬, 陈左宁, 张鲁飞, 等. 联邦学习中的隐私问题研究进展[J]. 软件学报, 2023, 34(1): 197–229. doi: 10.13328/j.cnki.jos.006411 TANG Lingtao, CHEN Zuoning, ZHANG Lufei, et al. Research progress of privacy issues in federated learning[J]. Journal of Software, 2023, 34(1): 197–229. doi: 10.13328/j.cnki.jos.006411
[10]	SO J, NOLET C J, YANG C S, et al. Lightsecagg: A lightweight and versatile design for secure aggregation in federated learning[C]. Proceedings of Machine Learning and Systems, Santa Clara, USA, 2022: 694–720.
[11]	FANG Minghong, LIU Jia, GONG N Z, et al. AFLGuard: Byzantine-robust asynchronous federated learning[C]. Proceedings of the 38th Annual Computer Security Applications Conference, Austin, USA, 2022: 632–646.
[12]	WANG Rong and TSAI W T. Asynchronous federated learning system based on permissioned blockchains[J]. Sensors, 2022, 22(4): 1672. doi: 10.3390/s22041672
[13]	LU Yunlong, HUANG Xiaohong, DAI Yueyue, et al. Differentially private asynchronous federated learning for mobile edge computing in urban informatics[J]. IEEE Transactions on Industrial Informatics, 2020, 16(3): 2134–2143. doi: 10.1109/TII.2019.2942179
[14]	DAMASKINOS G, EL MHAMDI E M, GUERRAOUI R, et al. Asynchronous Byzantine machine learning (the case of SGD)[C]. Proceedings of the 35th International Conference on Machine Learning, Stockholmsmässan, Sweden, 2018: 1153–1162.
[15]	刘艺璇, 陈红, 刘宇涵, 等. 联邦学习中的隐私保护技术[J]. 软件学报, 2022, 33(3): 1057–1092. doi: 10.13328/j.cnki.jos.006446 LIU Yixuan, CHEN Hong, LIU Yuhan, et al. Privacy-preserving techniques in federated learning[J]. Journal of Software, 2022, 33(3): 1057–1092. doi: 10.13328/j.cnki.jos.006446
[16]	WANG Bo, LI Hongtao, GUO Yina, et al. PPFLHE: A privacy-preserving federated learning scheme with homomorphic encryption for healthcare data[J]. Applied Soft Computing, 2023, 146: 110677. doi: 10.1016/j.asoc.2023.110677
[17]	FENG Jun, YANG L T, ZHU Qing, et al. Privacy-preserving tensor decomposition over encrypted data in a federated cloud environment[J]. IEEE Transactions on Dependable and Secure Computing, 2020, 17(4): 857–868. doi: 10.1109/TDSC.2018.2881452
[18]	李腾, 方保坤, 马卓, 等. 基于同态加密的医疗数据密文异常检测方法[J]. 中国科学:信息科学, 2023, 53(7): 1368–1391. doi: 10.1360/ssi-2022-0214 LI Teng, FANG Baokun, MA Zhuo, et al. Homomorphic encryption-based ciphertext anomaly detection method for e-health records[J]. Scientia Sinica (Informationis), 2023, 53(7): 1368–1391. doi: 10.1360/ssi-2022-0214
[19]	GEHLHAR T, MARX F, SCHNEIDER T, et al. SafeFL: MPC-friendly framework for private and robust federated learning[C]. 2023 IEEE Security and Privacy Workshops (SPW), San Francisco, USA, 2023: 69–76.
[20]	MANSOURI M, ÖNEN M, JABALLAH W B, et al. Sok: Secure aggregation based on cryptographic schemes for federated learning[J]. Proceedings on Privacy Enhancing Technologies, 2023, 2023(1): 140–157. doi: 10.56553/popets-2023-0009.doi:10.56553/popets-2023-0009
[21]	FENG Jun, YANG L T, NIE Xin, et al. Edge–cloud-aided differentially private tucker decomposition for cyber–physical–social systems[J]. IEEE Internet of Things Journal, 2022, 9(11): 8387–8396. doi: 10.1109/JIOT.2020.3004826
[22]	CAO Di, CHANG Shan, LIN Zhijian, et al. Understanding distributed poisoning attack in federated learning[C]. IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS), Tianjin, China, 2019: 233–239.
[23]	ABADI M, CHU A, GOODFELLOW I, et al. Deep learning with differential privacy[C]. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, 2016: 308–318.
[24]	KONEČNÝ J, MCMAHAN H B, XU F X, et al. Federated learning: Strategies for improving communication efficiency[C]. 6th International Conference on Learning Representations, Vancouver, BC, Canada, 2016.
[25]	BLANCHARD P, EL MHAMDI E M, GUERRAOUI R, et al. Machine learning with adversaries: Byzantine tolerant gradient descent[C]. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 118–128.
[26]	YIN Dong, CHEN Yudong, KANNAN R, et al. Byzantine-robust distributed learning: Towards optimal statistical rates[C]. Proceedings of the 35th International Conference on Machine Learning, Stockholmsmässan, Sweden, 2018: 5650–5659.