Research on Data Heterogeneous Robust Federated Learning with Privacy Protection in Internet of Things

YANG Zhigang; WANG Zhuotong; WU Dapeng; WANG Ruyan; WU Yu; LÜ Yi

doi:10.11999/JEIT221193

Volume 45 Issue 12

Dec. 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2023 > 45(12): 4235-4244

YANG Zhigang, WANG Zhuotong, WU Dapeng, WANG Ruyan, WU Yu, LÜ Yi. Research on Data Heterogeneous Robust Federated Learning with Privacy Protection in Internet of Things[J]. Journal of Electronics & Information Technology, 2023, 45(12): 4235-4244. doi: 10.11999/JEIT221193

Citation:

YANG Zhigang, WANG Zhuotong, WU Dapeng, WANG Ruyan, WU Yu, LÜ Yi. Research on Data Heterogeneous Robust Federated Learning with Privacy Protection in Internet of Things[J]. Journal of Electronics & Information Technology, 2023, 45(12): 4235-4244. doi: 10.11999/JEIT221193

Citation:

PDF( 8464 KB)

Research on Data Heterogeneous Robust Federated Learning with Privacy Protection in Internet of Things

doi: 10.11999/JEIT221193

YANG Zhigang^{1, 2, 3, 4},
WANG Zhuotong^{2, 3, 4},
WU Dapeng^{2, 3, 4
,
,},
WANG Ruyan^{2, 3, 4},
WU Yu¹,
LÜ Yi^{2, 3, 4}

1.
School of Cyber Security and Information Law, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
2.
School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
3.
Advanced Network and Intelligent Connection Technology Key Laboratory of Chongqing Education Commission of China, Chongqing 400065, China
4.
Chongqing Key Laboratory of Ubiquitous Sensing and Networking, Chongqing 400065, China

Funds: The National Natural Science Foundation of China (61901071, 61871062, 61771082, 62271096, U20A20157), The Natural Science Foundation of Chongqing (cstc2020jcyj-zdxmX0024), The University Innovation Research Group of Chongqing Foundation (CXQT20017), The Program for Innovation Team Building at Institutions of Higher Education in Chongqing (CXTDX201601020), The Youth Innovation Group Support Program of ICE Discipline of Chongqing University of Posts and Telecommunications (SCIE-QN-2022-04)

Received Date: 2022-09-14
Rev Recd Date: 2023-01-15

Available Online: 2023-02-08

Publish Date: 2023-12-26

Abstract

Abstract

Federated learning allows the effective flow of data value without leaving the local data, which is considered to be an effective way to balance data sharing and privacy protection in the Internet of Things (IoT) scenario. However, federated learning systems are vulnerable to Byzantine attacks and inference attacks, leading to the robustness of the system and the privacy of the data being compromised. The data heterogeneity and resource bottlenecks of IoT devices also pose significant challenges to the design of privacy-preserving and Byzantine-robust algorithms. In this paper, a data resampling of robust aggregation method Re-Sim applicable to heterogeneous IoT is proposed, which achieves robust aggregation by measuring directional similarity and normalized update magnitude, and uses data resampling to enhance robustness in data heterogeneous environments. Meanwhile, a Lightweight Security Aggregation (LSA) protocol is proposed to ensure data privacy while taking into account model robustness, accuracy and computational overhead, and the privacy of the protocol is theoretically analyzed. Simulation results show that the proposed scheme can effectively resist Byzantine attacks and inference attacks in the case of data heterogeneity. The proposed scheme improves the accuracy by 1%～3% compared to the baseline method, while reducing the client-side computational overhead by 79%.
- Internet of Things (IoT),
- Federated learning,
- Robust aggregation,
- Secret sharing

FullText(HTML)

References(20)

References

[1]	黄新林, 郑人华. 基于强化学习的802.11ax上行链路调度算法[J]. 电子与信息学报, 2022, 44(5): 1800–1808. doi: 10.11999/JEIT210590 HUANG Xinlin and ZHENG Renhua. 802.11ax uplink scheduling algorithm based on reinforcement learning[J]. Journal of Electronics &Information Technology, 2022, 44(5): 1800–1808. doi: 10.11999/JEIT210590
[2]	MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]. The 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA, 2017: 1273–1282.
[3]	BLANCHARD P, EL MHAMDI E M, GUERRAOUI R, et al. Machine learning with adversaries: Byzantine tolerant gradient descent[C]. The 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 118–128.
[4]	YIN Dong, CHEN Yudong, RAMCHANDRAN K, et al. Byzantine-robust distributed learning: Towards optimal statistical rates[C]. The 35th International Conference on Machine Learning, Stockholm, Sweden, 2018: 5636–5645.
[5]	MELIS L, SONG Congzheng, DE CRISTOFARO E, et al. Exploiting unintended feature leakage in collaborative learning[C]. 2019 IEEE Symposium on Security and Privacy (SP), San Francisco, USA, 2019: 691–706.
[6]	XIONG Jinbo, BI Renwan, TIAN Youliang, et al. Toward lightweight, privacy-preserving cooperative object classification for connected autonomous vehicles[J]. IEEE Internet of Things Journal, 2022, 9(4): 2787–2801. doi: 10.1109/JIOT.2021.3093573
[7]	BI Renwan, XIONG Jinbo, TIAN Youliang, et al. Achieving lightweight and privacy-preserving object detection for connected autonomous vehicles[J]. IEEE Internet of Things Journal, 2023, 10(3): 2314–2329. doi: 10.1109/JIOT.2022.3212464
[8]	ZAWAD S, ALI A, CHEN Pinyu, et al. Curse or redemption? How data heterogeneity affects the robustness of federated learning[C]. The 35th AAAI Conference on Artificial Intelligence, Vancouver, Canada, 2021: 10807–10814.
[9]	ZHAI Kun, REN Qiang, WANG Junli, et al. Byzantine-robust federated learning via credibility assessment on non-IID data[J]. arXiv: 2109.02396, 2021.
[10]	LI Liping, XU Wei, CHEN Tianyi, et al. RSA: Byzantine-robust stochastic aggregation methods for distributed learning from heterogeneous datasets[C]. The 33rd AAAI Conference on Artificial Intelligence, Hawaii, USA, 2019: 1544–1551.
[11]	ZHANG Chengliang, LI Suyi, XIA Junzhe, et al. BatchCrypt: Efficient homomorphic encryption for cross-silo federated learning[C]. The 2020 USENIX Conference on Usenix Annual Technical Conference, Boston, USA, 2020: 33.
[12]	FU Anmin, ZHANG Xianglong, XIONG Naixue, et al. VFL: A verifiable federated learning with privacy-preserving for big data in industrial IoT[J]. IEEE Transactions on Industrial Informatics, 2022, 18(5): 3316–3326. doi: 10.1109/TII.2020.3036166
[13]	SO J, GÜLER B, and AVESTIMEHR A S. Byzantine-resilient secure federated learning[J]. IEEE Journal on Selected Areas in Communications, 2021, 39(7): 2168–2181. doi: 10.1109/JSAC.2020.3041404
[14]	LIU Xiaoyuan, LI Hongwei, XU Guowen, et al. Privacy-enhanced federated learning against poisoning adversaries[J]. IEEE Transactions on Information Forensics and Security, 2021, 16: 4574–4588. doi: 10.1109/TIFS.2021.3108434
[15]	HSIEH K, HARLAP A, VIJAYKUMAR N, et al. Gaia: Geo-distributed machine learning approaching LAN speeds[C]. The 14th USENIX Conference on Networked Systems Design and Implementation, Boston, USA, 2017: 629–647.
[16]	LI Yiran, LI Hongwei, XU Guowen, et al. Efficient privacy-preserving federated learning with unreliable users[J]. IEEE Internet of Things Journal, 2021, 9(13): 11590–11603. doi: 10.1109/JIOT.2021.3130115
[17]	ZHU Wanchuang, ZHAO B Z H, LUO S, et al. MANDERA: Malicious node detection in federated learning via ranking[J]. arXiv: 2110.11736, 2021.
[18]	LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278–2324. doi: 10.1109/5.726791
[19]	XIAO Han, RASUL K, and VOLLGRAF R. Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms[J]. arXiv: 1708.07747, 2017.
[20]	WANG Hao, KAPLAN Z, NIU Di, et al. Optimizing federated learning on non-IID data with reinforcement learning[C]. IEEE INFOCOM 2020 - IEEE Conference on Computer Communications, Toronto, Canada, 2020: 1698–1707.