Research on Efficient Federated Learning Communication Mechanism Based on Adaptive Gradient Compression

TANG Lun; WANG Zhiping; PU Hao; WU Zhuang; CHEN Qianbin

doi:10.11999/JEIT211262

Volume 45 Issue 1

Jan. 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2023 > 45(1): 227-234

TANG Lun, WANG Zhiping, PU Hao, WU Zhuang, CHEN Qianbin. Research on Efficient Federated Learning Communication Mechanism Based on Adaptive Gradient Compression[J]. Journal of Electronics & Information Technology, 2023, 45(1): 227-234. doi: 10.11999/JEIT211262

Citation:

TANG Lun, WANG Zhiping, PU Hao, WU Zhuang, CHEN Qianbin. Research on Efficient Federated Learning Communication Mechanism Based on Adaptive Gradient Compression[J]. Journal of Electronics & Information Technology, 2023, 45(1): 227-234. doi: 10.11999/JEIT211262

Citation:

PDF( 5184 KB)

Research on Efficient Federated Learning Communication Mechanism Based on Adaptive Gradient Compression

doi: 10.11999/JEIT211262

1.
School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
2.
Key Laboratory of Mobile Communications Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

Funds: The National Natural Science Foundation of China (62071078), The Science and Technology Research Program of Chongqing Municipal Education Commission (KJZD-M201800601), Sichuan and Chongqing Key R&D Projects (2021YFQ0053)

Received Date: 2021-11-12
Rev Recd Date: 2022-04-22

Available Online: 2022-04-28

Publish Date: 2023-01-17

Abstract

Abstract

Considering the non-negligible communication cost problem caused by redundant gradient interactive communication between a large number of device nodes in the Federated Learning(FL) process in the Internet of Things (IoTs) scenario, gradient communication compression mechanism with adaptive threshold is proposed. Firstly, a structure of Communication-Efficient EDge-Federated Learning (CE-EDFL) is used to prevent device-side data privacy leakage. The edge server acts as an intermediary device to perform device-side local model aggregation, and the cloud performs edge server model aggregation and new parameter delivery. Secondly, in order to reduce further the communication overhead during federated learning detection, a threshold Adaptive Lazily Aggregated Gradient (ALAG) is proposed, which reduces the redundant communication between the device end and the edge server by compressing the gradient parameters of the local model. The experimental results show that the proposed algorithm can effectively improve the overall communication efficiency of the model by reducing the number of gradient interactions while ensuring the accuracy of deep learning tasks in the large-scale IoT device scenario.
- Federated Learning(FL),
- Edge computing,
- Communication optimization,
- Gradient compression

FullText(HTML)

References(14)

References

[1]	LI Tian, SAHU A K, TALWALKAR A, et al. Federated learning: Challenges, methods, and future directions[J]. IEEE Signal Processing Magazine, 2020, 37(3): 50–60. doi: 10.1109/MSP.2020.2975749
[2]	LUO Siqi, CHEN Xu, WU Qiong, et al. HFEL: Joint edge association and resource allocation for cost-efficient hierarchical federated edge learning[J]. IEEE Transactions on Wireless Communications, 2020, 19(10): 6535–6548. doi: 10.1109/TWC.2020.3003744
[3]	HUANG Liang, FENG Xu, FENG Anqi, et al. Distributed deep learning-based offloading for mobile edge computing networks[J]. Mobile Networks and Applications, 2022, 27: 1123–1130. doi: 10.1007/s11036-018-1177-x
[4]	赵英, 王丽宝, 陈骏君, 等. 基于联邦学习的网络异常检测[J]. 北京化工大学学报:自然科学版, 2021, 48(2): 92–99. doi: 10.13543/j.bhxbzr.2021.02.012 ZHAO Ying, WANG Libao, CHEN Junjun, et al. Network anomaly detection based on federated learning[J]. Journal of Beijing University of Chemical Technology:Natural Science, 2021, 48(2): 92–99. doi: 10.13543/j.bhxbzr.2021.02.012
[5]	周传鑫, 孙奕, 汪德刚, 等. 联邦学习研究综述[J]. 网络与信息安全学报, 2021, 7(5): 77–92. doi: 10.11959/j.issn.2096-109x.2021056 ZHOU Chuanxin, SUN Yi, WANG Degang, et al. Survey of federated learning research[J]. Chinese Journal of Network and Information Security, 2021, 7(5): 77–92. doi: 10.11959/j.issn.2096-109x.2021056
[6]	SHI Weisong, CAO Jie, ZHANG Quan, et al. Edge computing: Vision and challenges[J]. IEEE Internet of Things Journal, 2016, 3(5): 637–646. doi: 10.1109/JIOT.2016.2579198
[7]	WANG Shiqiang, TUOR T, SALONIDIS T, et al. Adaptive federated learning in resource constrained edge computing systems[J]. IEEE Journal on Selected Areas in Communications, 2019, 37(6): 1205–1221. doi: 10.1109/JSAC.2019.2904348
[8]	ABESHU A and CHILAMKURTI N. Deep learning: The frontier for distributed attack detection in fog-to-things computing[J]. IEEE Communications Magazine, 2018, 56(2): 169–175. doi: 10.1109/MCOM.2018.1700332
[9]	LIU Lumin, ZHANG Jun, SONG Shenghui, et al. Client-edge-cloud hierarchical federated learning[C]. ICC 2020 - 2020 IEEE International Conference on Communications (ICC), Dublin, Ireland, 2020: 1–6.
[10]	SATTLER F, WIEDEMANN S, MÜLLER K R, et al. Robust and communication-efficient federated learning from Non-i. i. d. data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(9): 3400–3413. doi: 10.1109/TNNLS.2019.2944481
[11]	SUN Jun, CHEN Tianyi, GIANNAKIS G B, et al. Lazily aggregated quantized gradient innovation for communication-efficient federated learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(4): 2031–2044. doi: 10.1109/TPAMI.2020.3033286
[12]	CHEN Tianyi, SUN Yuejiao, and YIN Wotao. Communication-adaptive stochastic gradient methods for distributed learning[J]. IEEE Transactions on Signal Processing, 2021, 69: 4637–4651. doi: 10.1109/TSP.2021.3099977
[13]	LU Xiaofeng, LIAO Yuying, LIO P, et al. Privacy-preserving asynchronous federated learning mechanism for edge network computing[J]. IEEE Access, 2020, 8: 48970–48981. doi: 10.1109/ACCESS.2020.2978082
[14]	MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]. The 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA, 2016: 1273–1282.