Adaptively Sparse Federated Learning Optimization Algorithm Based on Edge-assisted Server

CHEN Xiao; QIU Hongbing; LI Yanlong

doi:10.11999/JEIT240741

Volume 47 Issue 3

Mar. 2025

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 > 47(3): 645-656

Liu Zheng-jun, Zou Xi, Ran Chong-sen. Twice-Correlate Rapid Acquisition Algorithm for Synchronization of PRACH Preamble in WCDMA Reverse Link[J]. Journal of Electronics & Information Technology, 2004, 26(8): 1262-1268.

Citation:

CHEN Xiao, QIU Hongbing, LI Yanlong. Adaptively Sparse Federated Learning Optimization Algorithm Based on Edge-assisted Server[J]. Journal of Electronics & Information Technology, 2025, 47(3): 645-656. doi: 10.11999/JEIT240741

Citation:

PDF( 2861 KB)

Adaptively Sparse Federated Learning Optimization Algorithm Based on Edge-assisted Server

doi: 10.11999/JEIT240741

School of Information and Communication, Guilin University of Electronic Technology, Guilin 541004, China

Funds: The National Natural Science Foundation of China (61571143), The Innovation Project of Guangxi Graduate Education (YCBZ2022106)

Received Date: 2024-08-28
Rev Recd Date: 2024-12-29

Available Online: 2025-01-13

Publish Date: 2025-03-01

Abstract

Abstract

Objective Federated Learning (FL) represents a distributed learning framework with significant potential, allowing users to collaboratively train a shared model while retaining data on their devices. However, the substantial differences in computing, storage, and communication capacities across FL devices within complex networks result in notable disparities in model training and transmission latency. As communication rounds increase, a growing number of heterogeneous devices become stragglers due to constraints such as limited energy and computing power, changes in user intentions, and dynamic channel fluctuations, adversely affecting system convergence performance. This study addresses these challenges by jointly incorporating assistance mechanisms and reducing device overhead to mitigate the impact of stragglers on model accuracy and training latency. Methods This paper designs an FL architecture integrating joint edge-assisted training and adaptive sparsity and proposes an adaptively sparse FL optimization algorithm based on edge-assisted training. First, an edge server is introduced to provide auxiliary training for devices with limited computing power or energy. This reduces the training delay of the FL system, enables stragglers to continue participating in the training process, and helps maintain model accuracy. Specifically, an optimization model for auxiliary training, communication, and computing resource allocation is constructed. Several deep reinforcement learning methods are then applied to obtain the optimized auxiliary training decision. Second, based on the auxiliary training decision, unstructured pruning is adaptively performed on the global model during each communication round to further reduce device delay and energy consumption. Results and Discussions The proposed framework and algorithm are evaluated through extensive simulations. The results demonstrate the effectiveness and efficiency of the proposed method in terms of model accuracy and training delay.The proposed algorithm achieves an accuracy rate approximately 5% higher than that of the FL algorithm on both the MNIST and CIFAR-10 datasets. This improvement results from low-computing-power and low-energy devices failing to transmit their local models to the central server during multiple communication rounds, reducing the global model’s accuracy (Table 3).The proposed algorithm achieves an accuracy rate 18% higher than that of the FL algorithm on the MNIST-10 dataset when the data on each device follow a non-IID distribution. Statistical heterogeneity exacerbates model degradation caused by stragglers, whereas the proposed algorithm significantly improves model accuracy under such conditions (Table 4).The reward curves of different algorithms are presented (Fig. 7). The reward of FL remains constant, while the reward of EAFL_RANDOM fluctuates randomly. ASEAFL_DDPG shows a more stable reward curve once training episodes exceed 120 due to the strong learning and decision-making capabilities of DDPG and DQN. In contrast, EAFL_DQN converges more slowly and maintains a lower reward than the proposed algorithm, mainly due to more precise decision-making in the continuous action space and an exploration mechanism that expands action selection (Fig. 7).When the computing power of the edge server increases, the training delay of the FL algorithm remains constant since it does not involve auxiliary training. The training delay of EAFL_RANDOM fluctuates randomly, while the delays of ASEAFL_DDPG and EAFL_DQN decrease. However, ASEAFL_DDPG consistently achieves a lower system training delay than EAFL_DQN under the same MEC computing power conditions (Fig. 9).When the communication bandwidth between the edge server and devices increases, the training delay of the FL algorithm remains unchanged as it does not involve auxiliary training. The training delay of EAFL_RANDOM fluctuates randomly, while the delays of ASEAFL_DDPG and EAFL_DQN decrease. ASEAFL_DDPG consistently achieves lower system training delay than EAFL_DQN under the same bandwidth conditions (Fig. 10). Conclusions The proposed sparse-adaptive FL architecture based on an edge-assisted server mitigates the straggler problem caused by system heterogeneity from two perspectives. By reducing the number of stragglers, the proposed algorithm achieves higher model accuracy compared with the traditional FL algorithm, effectively decreases system training delay, and improves model training efficiency. This framework holds practical value, particularly for FL deployments where aggregation devices are selected based on statistical characteristics, such as model contribution rates. Straggler issues are common in such FL scenarios, and the proposed architecture effectively reduces their occurrence. Simultaneously, devices with high model contribution rates can continue participating in multiple rounds of federated training, lowering the central server’s frequent device selection overhead. Additionally, in resource-constrained FL environments, edge servers can perform more diverse and flexible tasks, such as partial auxiliary training and partitioned model training.
- Federated Learning (FL),
- Edge server,
- Adaptive sparsity,
- Deep reinforcement learning,
- Unstructured pruning

FullText(HTML)

References(27)

References

[1]	CHENG Nan, WU Shen, WANG Xiucheng, et al. AI for UAV-assisted IoT applications: A comprehensive review[J]. IEEE Internet of Things Journal, 2023, 10(16): 14438–14461. doi: 10.1109/JIOT.2023.3268316.
[2]	ALSELEK M, ALCARAZ-CALERO J M, and WANG Qi. Dynamic AI-IoT: Enabling updatable AI models in ultralow-power 5G IoT devices[J]. IEEE Internet of Things Journal, 2024, 11(8): 14192–14205. doi: 10.1109/JIOT.2023.3340858.
[3]	KALAKOTI R, BAHSI H, and NÕMM S. Improving IoT security with explainable AI: Quantitative evaluation of explainability for IoT botnet detection[J]. IEEE Internet of Things Journal, 2024, 11(10): 18237–18254. doi: 10.1109/JIOT.2024.3360626.
[4]	KUMAR R, JAVEED D, ALJUHANI A, et al. Blockchain-based authentication and explainable AI for securing consumer IoT applications[J]. IEEE Transactions on Consumer Electronics, 2024, 70(1): 1145–1154. doi: 10.1109/TCE.2023.3320157.
[5]	MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]. The 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA, 2017: 1273–1282.
[6]	LI Xingyu, QU Zhe, TANG Bo, et al. Stragglers are not disasters: A hybrid federated learning framework with delayed gradients[C]. The 21st IEEE International Conference on Machine Learning and Applications (ICMLA), Nassau, Bahamas, 2022: 727–732. doi: 10.1109/ICMLA55696.2022.00121.
[7]	LIANG Kai and WU Youlong. Two-layer coded gradient aggregation with straggling communication links[C]. 2020 IEEE Information Theory Workshop (ITW), Riva del Garda, Italy, 2021: 1–5. doi: 10.1109/ITW46852.2021.9457626.
[8]	LANG N, COHEN A, and SHLEZINGER N. Stragglers-aware low-latency synchronous federated learning via layer-wise model updates[J]. arXiv: 2403.18375, 2024. doi: 10.48550/arXiv.2403.18375.
[9]	MHAISEN N, ABDELLATIF A A, MOHAMED A, et al. Optimal user-edge assignment in hierarchical federated learning based on statistical properties and network topology constraints[J]. IEEE Transactions on Network Science and Engineering, 2022, 9(1): 55–66. doi: 10.1109/TNSE.2021.3053588.
[10]	FENG Chenyuan, YANG H H, HU Deshun, et al. Mobility-aware cluster federated learning in hierarchical wireless networks[J]. IEEE Transactions on Wireless Communications, 2022, 21(10): 8441–8458. doi: 10.1109/TWC.2022.3166386.
[11]	LIM W Y B, NG J S, XIONG Zehui, et al. Decentralized edge intelligence: A dynamic resource allocation framework for hierarchical federated learning[J]. IEEE Transactions on Parallel and Distributed Systems, 2022, 33(3): 536–550. doi: 10.1109/TPDS.2021.3096076.
[12]	KONG J M and SOUSA E. Adaptive ratio-based-threshold gradient sparsification scheme for federated learning[C]. 2023 International Symposium on Networks, Computers and Communications (ISNCC), Doha, Qatar, 2023: 1–5. doi: 10.1109/ISNCC58260.2023.10323644.
[13]	SU Junshen, WANG Xijun, CHEN Xiang, et al. Joint sparsification and quantization for wireless federated learning under communication constraints[C]. 2023 IEEE 24th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Shanghai, China, 2023: 401–405. doi: 10.1109/SPAWC53906.2023.10304559.
[14]	PARK S and CHOI W. Regulated subspace projection based local model update compression for communication-efficient federated learning[J]. IEEE Journal on Selected Areas in Communications, 2023, 41(4): 964–976. doi: 10.1109/JSAC.2023.3242722.
[15]	DHAKAL S, PRAKASH S, YONA Y, et al. Coded federated learning[C]. 2019 IEEE Globecom Workshops (GC Wkshps), Waikoloa, USA, 2019: 1–6. doi: 10.1109/GCWkshps45667.2019.9024521.
[16]	PRAKASH S, DHAKAL S, AKDENIZ M R, et al. Coded computing for low-latency federated learning over wireless edge networks[J]. IEEE Journal on Selected Areas in Communications, 2021, 39(1): 233–250. doi: 10.1109/JSAC.2020.3036961.
[17]	SUN Yuchang, SHAO Jiawei, MAO Yuyi, et al. Stochastic coded federated learning: Theoretical analysis and incentive mechanism design[J]. IEEE Transactions on Wireless Communications, 2024, 23(6): 6623–6638. doi: 10.1109/TWC.2023.3334732.
[18]	BANERJEE S, VU X S, and BHUYAN M. Optimized and adaptive federated learning for straggler-resilient device selection[C]. 2022 International Joint Conference on Neural Networks (IJCNN), Padua, Italy, 2022: 1–9. doi: 10.1109/IJCNN55064.2022.9892777.
[19]	HUANG Peishan, LI Dong, and YAN Zhigang. Wireless federated learning with asynchronous and quantized updates[J]. IEEE Communications Letters, 2023, 27(9): 2393–2397. doi: 10.1109/LCOMM.2023.3294606.
[20]	YAN Xinru, MIAO Yinbin, LI Xinghua, et al. Privacy-preserving asynchronous federated learning framework in distributed IoT[J]. IEEE Internet of Things Journal, 2023, 10(15): 13281–13291. doi: 10.1109/JIOT.2023.3262546.
[21]	YANG Zhigang, ZHANG Xuhua, WU Dapeng, et al. Efficient asynchronous federated learning research in the internet of vehicles[J]. IEEE Internet of Things Journal, 2023, 10(9): 7737–7748. doi: 10.1109/JIOT.2022.3230412.
[22]	DIAO E, DING Jie, and TAROKH V. HeteroFL: Computation and communication efficient federated learning for heterogeneous clients[C]. 9th International Conference on Learning Representations, 2021.
[23]	AL-ABIAD M S, HASSAN M Z, and HOSSAIN M J. Energy-efficient resource allocation for federated learning in NOMA-enabled and relay-assisted internet of things networks[J]. IEEE Internet of Things Journal, 2022, 9(24): 24736–24753. doi: 10.1109/JIOT.2022.3194546.
[24]	TANG Jianhang, NIE Jiangtian, ZHANG Yang, et al. Multi-UAV-assisted federated learning for energy-aware distributed edge training[J]. IEEE Transactions on Network and Service Management, 2024, 21(1): 280–294. doi: 10.1109/TNSM.2023.3298220.
[25]	LI Yuchen, LIANG Weifa, LI Jing, et al. Energy-aware, device-to-device assisted federated learning in edge computing[J]. IEEE Transactions on Parallel and Distributed Systems, 2023, 34(7): 2138–2154. doi: 10.1109/TPDS.2023.3277423.
[26]	高晗, 田育龙, 许封元, 等. 深度学习模型压缩与加速综述[J]. 软件学报, 2021, 32(1): 68–92. doi: 10.13328/j.cnki.jos.006096. GAO Han, TIAN Yulong, XU Fengyuan, et al. Survey of deep learning model compression and acceleration[J]. Journal of Software, 2021, 32(1): 68–92. doi: 10.13328/j.cnki.jos.006096.
[27]	STRIPELIS D, GUPTA U, VER STEEG G, et al. Federated progressive sparsification (purge, merge, tune)+[J]. arXiv: 2204.12430, 2022. doi: 10.48550/arXiv.2204.12430.

Relative Articles

Supplements(0)

Cited By

Cited by

Periodical cited type(14)

1.	张盛峰，袁强，陈会丹，黄胜. EONs中一种混合路径专有保护算法. 光通信研究. 2021(02): 20-25 .
2.	胡竣涛，时小虎，马德印. 基于均值漂移和遗传算法的护工调度算法. 广西师范大学学报(自然科学版). 2021(03): 27-39 .
3.	巨子琪，兰宏伟，宰晨光. 轨道交通车辆踏面制动闸调器螺杆连接优化研究. 自动化与仪器仪表. 2020(12): 70-74 .
4.	赵必游，张善辉，王进帅. 配电系统弹性光网络频谱整理优化算法. 电信科学. 2019(02): 43-50 .
5.	黄正鹏，王力，张仕学，余廷忠，张起荣. 基于传统遗传和数据压缩算法的冗余光纤数据存储优化. 激光杂志. 2019(03): 135-139 .
6.	程光德，肖瑜. 基于用户满意度的光网络数据路由机制设计. 激光杂志. 2019(04): 118-121 .
7.	王鹏辉，张宁，肖明明. 基于节点重要度的路由选择与频谱分配算法. 计算机工程与应用. 2019(13): 106-111+259 .
8.	马学森，朱建，谈杰，唐昊，周江涛. 多头绒泡菌预处理的改进Q学习算法求解最短路径问题. 电子测量与仪器学报. 2019(05): 148-157 .
9.	施达雅，余庚. 弹性光网络中碎片问题的研究. 光通信技术. 2018(02): 16-19 .
10.	田建勇，石林江. 基于Kalman算法的光纤网络流量在线预测模型. 激光杂志. 2018(09): 110-114 .
11.	刘焕淋，胡浩，熊翠连，陈勇，向敏，马跃. 基于时频联合碎片感知的资源均衡虚拟光网络映射算法. 电子与信息学报. 2018(10): 2345-2351 . 本站查看
12.	刘焕淋，张明佳，陈勇，王欣. 频谱可用性和保护带宽共享度感知的弹性光网络生存性多路径策略. 电子与信息学报. 2017(10): 2472-2478 . 本站查看
13.	李汪丽. 通信链接无线终端资源传输路径目标识别仿真. 计算机仿真. 2017(10): 177-180 .
14.	魏星. 基于改进人工鱼群算法的光网络最优环路径搜索研究. 计算机与数字工程. 2017(04): 650-654 .