Personalized Federated Learning Method Based on Collation Game and Knowledge Distillation

SUN Yanhua; SHI Yahui; LI Meng; YANG Ruizhe; SI Pengbo

doi:10.11999/JEIT221203

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2023 >

CHEN Tao, WANG Mengxin, HUANG Xiangsong. Time Difference of Arrival Passive Location Based on Salp Swarm Algorithm[J]. Journal of Electronics & Information Technology, 2018, 40(7): 1591-1597. doi: 10.11999/JEIT170979

Citation:

SUN Yanhua, SHI Yahui, LI Meng, YANG Ruizhe, SI Pengbo. Personalized Federated Learning Method Based on Collation Game and Knowledge Distillation[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT221203

Citation:

PDF( 1097 KB)

Personalized Federated Learning Method Based on Collation Game and Knowledge Distillation

doi: 10.11999/JEIT221203

SUN Yanhua^{1, 2
,
,},
SHI Yahui^{1, 2},
LI Meng^{1, 2},
YANG Ruizhe^{1, 2},
SI Pengbo^{1, 2}

1.
Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
2.
Beijing Laboratory of Advanced Information Networks, Beijing University of Technology, Beijing 100124, China

Funds: Foundation of Beijing Municipal Commission of Education (KM202010005017)

Received Date: 2022-09-20
Rev Recd Date: 2022-12-27

Available Online: 2022-12-28

Abstract

Abstract

To overcome the limitation of the Federated Learning (FL) when the data and model of each client are all heterogenous and improve the accuracy, a personalized Federated learning algorithm with Collation game and Knowledge distillation (pFedCK) is proposed. Firstly, each client uploads its soft-predict on public dataset and download the most correlative of the k soft-predict. Then, this method apply the shapley value from collation game to measure the multi-wise influences among clients and quantify their marginal contribution to others on personalized learning performance. Lastly, each client identify it’s optimal coalition and then distill the knowledge to local model and train on private dataset. The results show that compared with the state-of-the-art algorithm, this approach can achieve superior personalized accuracy and can improve by about 10%.
- Personalized Federated Learning(PFL),
- Collation game,
- Knowledge distillation,
- Heterogeneity

FullText(HTML)

References(15)

References

[1]	MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA, 2017: 1273–1282.
[2]	LI Tian, SAHU A K, TALWALKAR A, et al. Federated learning: Challenges, methods, and future directions[J]. IEEE Signal Processing Magazine, 2020, 37(3): 50–60. doi: 10.1109/MSP.2020.2975749
[3]	ARIVAZHAGAN M G, AGGARWAL V, SINGH A K, et al. Federated learning with personalization layers[EB/OL]. https://doi.org/10.48550/arXiv.1912.00818, 2019.
[4]	HANZELY F and RICHTÁRIK P. Federated learning of a mixture of global and local models[EB/OL]. https://doi.org/10.48550/arXiv.2002.05516, 2020.
[5]	FALLAH A, MOKHTARI A, and OZDAGLAR A E. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach[C]. Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2020: 3557–3568.
[6]	GHOSH A, CHUNG J, YIN D, et al. An efficient framework for clustered federated learning[C]. Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2020: 19586–19597.
[7]	WU Leijie, GUO Song, DING Yaohong, et al. A coalition formation game approach for personalized federated learning[EB/OL]. https://doi.org/10.48550/arXiv.2202.02502, 2022.
[8]	HINTON G, VINYALS O, and DEAN J. Distilling the knowledge in a neural network[J]. Computer Science, 2015, 14(7): 38–39.
[9]	LI Daliang and WANG Junpu. FedMD: Heterogenous federated learning via model distillation[EB/OL]. https://doi.org/10.48550/arXiv.1910.03581, 2019.
[10]	LIN Tao, KONG Lingjing, STICH S U, et al. Ensemble distillation for robust model fusion in federated learning[C]. Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2020: 2351–2363.
[11]	ZHANG Jie, GUO Song, MA Xiaosong, et al. Parameterized knowledge transfer for personalized federated learning[C]. Proceedings of the 35th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2021: 10092–10104.
[12]	CHO Y J, WANG Jianyu, CHIRUVOLU T, et al. Personalized federated learning for heterogeneous clients with clustered knowledge transfer[EB/OL]. https://doi.org/10.48550/arXiv.2109.08119, 2021.
[13]	DONAHUE K and KLEINBERG J. Model-sharing games: Analyzing federated learning under voluntary participation[C]. Proceedings of the 35th AAAI Conference on Artificial Intelligence, Vancouver, Canada, 2021: 5303–5311.
[14]	LUNDBERG S M and LEE S I. A unified approach to interpreting model predictions[C]. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 4768–4777.
[15]	SAAD W, HAN Zhu, DEBBAH M, et al. Coalitional game theory for communication networks[J]. IEEE Signal Processing Magazine, 2009, 26(5): 77–97. doi: 10.1109/MSP.2009.000000

Relative Articles

Supplements(0)

Cited By

Cited by

Periodical cited type(49)

1.	张宏，周大伟，陆丽，康小东. 基于改进的樽海鞘群算法的永磁同步电机多目标优化设计. 电机与控制应用. 2025(02): 221-230 .
2.	徐良，田青云，文成，张海波，郭晶晶. 基于改进樽海鞘群算法的测试数据自动生成. 信息技术与信息化. 2024(01): 117-121 .
3.	李辉，殷文明. 探索者变异樽海鞘算法及其应用. 数学的实践与认识. 2024(03): 151-159 .
4.	蒙淑娇，晋良念. 基于改进鲸鱼优化的地面短基线单站无源定位方法. 无线电工程. 2024(07): 1739-1748 .
5.	曹宁，严心娥，徐根祺，许又文，张正勃，杜倩云. 基于DEFA-LSSAR的水利工程边坡力学参数预测模型. 计算机与现代化. 2024(07): 106-111 .
6.	史红伟，左越. 基于LPNN的无源ML-TDOA估计. 沈阳工业大学学报. 2024(06): 832-839 .
7.	肖剑，刘经纬，胡欣，齐小刚. 基于改进非洲秃鹫算法的TDOA-AOA定位. 吉林大学学报(工学版). 2024(12): 3558-3567 .
8.	陈一馨，张婷，刘永刚，陈晶. 基于改进樽海鞘群算法的提梁机主梁轻量化设计方法. 东北大学学报(自然科学版). 2023(02): 223-232 .
9.	牛昊一，吴维敏，章庭棋，沈微，张涛 . 自适应樽海鞘群算法求解考虑运输时间的柔性作业车间调度. 浙江大学学报(工学版). 2023(07): 1267-1277 .
10.	问轲，林晶，张学昌，刘永跃. 混沌策略和非线性收敛因子的核参数寻优算法. 机械科学与技术. 2023(09): 1490-1501 .
11.	丁美芳，吴克晴，肖鹏. 多策略融合的黄金正弦樽海鞘群算法. 南京信息工程大学学报(自然科学版). 2023(06): 662-675 .
12.	叶智慧，吴红梅，王佩，熊伟，郭颖，陈略，董志源. 基于量子海鸥算法的运载火箭回收舱段时差定位方法. 上海航天(中英文). 2023(06): 121-135 .
13.	张铸，张仕杰，饶盛华，王静袁. 基于自适应正态云模型的引力樽海鞘群算法. 控制与决策. 2022(02): 344-352 .
14.	韦子辉，王世昭，叶兴跃，马英杰，李小阳，方立德. 基于超宽带的TDOA相邻单元协同定位技术. 电子测量技术. 2022(01): 77-83 .
15.	赵玉超，袁宏拓，孙铭. 基于单步加权最小二乘的战场集结定位算法. 河北科技大学学报. 2022(01): 42-49 .
16.	高岳林，杨钦文，王晓峰，李嘉航，宋彦杰. 新型群体智能优化算法综述. 郑州大学学报(工学版). 2022(03): 21-30 .
17.	马一鸣，石志东，赵康，贡常磊，单联海. 基于改进樽海鞘群算法的到达时间差定位. 上海大学学报(自然科学版). 2022(02): 238-249 .
18.	段绍米，罗会龙，刘海鹏. 人群搜索和樽海鞘群的混合算法优化PID参数. 系统仿真学报. 2022(06): 1230-1246 .
19.	方立德，王世昭，解云龙，李萌旭，韦子辉. 基于改进粒子群的TDOA三维定位解算方法. 现代电子技术. 2022(13): 45-50 .
20.	汤安迪，韩统，徐登武，周欢，谢磊. 使用高斯分布估计策略的改进樽海鞘群算法. 系统工程与电子技术. 2022(07): 2229-2240 .
21.	彭石燕，郑洪清. 复数编码的樽海鞘群算法及其应用. 广西民族大学学报(自然科学版). 2022(02): 81-86 .
22.	范纯龙，童航. 樽海鞘算法在测试用例约简问题中的应用. 电脑编程技巧与维护. 2022(10): 8-10+15 .
23.	余修武，黄露平，刘永，李佩. 融合柯西折射反向学习和变螺旋策略的WSN象群定位算法. 控制与决策. 2022(12): 3183-3189 .
24.	刘树东，梁婷蓉，王燕，张艳. 一种提高水下目标被动定位性能的两步定位法. 天津城建大学学报. 2022(06): 460-466 .
25.	刘小龙，许岩，徐维军. 基于统计引导和多项式差分学习的樽海鞘优化算法. 运筹与管理. 2021(01): 43-49 .
26.	王振亚，姚立纲，蔡永武，张俊. 基于熵-流特征和樽海鞘群优化支持向量机的故障诊断方法. 振动与冲击. 2021(06): 107-114 .
27.	刘景森，袁蒙蒙，李煜. 基于改进樽海鞘群算法求解工程优化设计问题. 系统仿真学报. 2021(04): 854-866 .
28.	魏鹏飞，樊小朝，史瑞静，王维庆，闫亚东. 基于互补式集合经验模态分解和SSA-ELM的短期风电功率预测. 水力发电. 2021(05): 116-120 .
29.	赵忠凯，刘楯，黄湘松. 无人机编队时差定位时的空间布局分析. 应用科技. 2021(02): 12-18+41 .
30.	韩超杰，郝玉然，刘亚飞. 基于SSA-Otsu的彩色图像多阈值分割研究. 现代计算机. 2021(10): 108-111 .
31.	陈连兴，牟永敏. 一种改进的樽海鞘群算法. 计算机应用研究. 2021(06): 1648-1652 .
32.	蒋美琪，杨兴，罗聪敏. 基于反向学习与混合位置中心的樽海鞘算法. 西华大学学报(自然科学版). 2021(01): 17-21+74 .
33.	杨兴，郭明昊，方霞，祝忠明，蒋美琪. 基于天牛须搜索自适应的樽海鞘算法. 计算机技术与发展. 2021(06): 1-6 .
34.	唐菁敏，郑锦文，曲文博. 基于改进自适应乌鸦搜索算法的无源定位. 重庆邮电大学学报(自然科学版). 2021(03): 372-377 .
35.	卓然，王未卿. 混沌映射与动态学习的自适应樽海鞘群算法. 计算机工程与设计. 2021(07): 1963-1972 .
36.	刘景森，袁蒙蒙，左方. 面向全局搜索的自适应领导者樽海鞘群算法. 控制与决策. 2021(09): 2152-2160 .
37.	田洪舟，陈思溢，黄辉先. 基于改进樽海鞘群算法的无线传感器网络节点定位. 传感器与微系统. 2021(09): 139-141+145 .
38.	唐铁斌，刘炜. 基于改进樽海鞘群算法的SDN控制器部署算法. 计算机应用与软件. 2021(12): 291-297 .
39.	孙光才，王裕旗，高昭昭，江帆，邢孟道，保铮. 一种基于短合成孔径的双星干涉精确定位方法. 电子与信息学报. 2020(02): 472-479 . 本站查看
40.	张文彬. 基于天体运动更新机制的改进樽海鞘群算法. 上海电力大学学报. 2020(02): 195-200 .
41.	陈忠云，张达敏，辛梓芸，张绘娟，闫威. 混沌精英质心拉伸机制的樽海鞘群算法. 计算机工程与应用. 2020(10): 44-50 .
42.	林国营，卢世祥，郭昆健，高赐威，冯小峰. 基于主从博弈的电网公司需求响应补贴定价机制. 电力系统自动化. 2020(10): 59-68 .
43.	王明超，董佳圆，李继影，高磊，聂永辉. 基于ISSA的STATCOM模型参数解耦辨识研究. 东北电力大学学报. 2020(01): 81-89 .
44.	张达敏，陈忠云，辛梓芸，张绘娟，闫威. 基于疯狂自适应的樽海鞘群算法. 控制与决策. 2020(09): 2112-2120 .
45.	陈忠云，张达敏，辛梓芸. 正弦余弦算法的樽海鞘群算法. 计算机应用与软件. 2020(09): 209-214 .
46.	范千，陈振健，夏樟华. 一种基于折射反向学习机制与自适应控制因子的改进樽海鞘群算法. 哈尔滨工业大学学报. 2020(10): 183-191 .
47.	马一鸣，石志东，赵康，贡常磊，单联海. 基于改进哈里斯鹰优化算法的TDOA定位. 计算机工程. 2020(12): 179-184 .
48.	孙铭阳. 基于SSA-PNN的矿井提升机主轴装置故障诊断. 无线互联科技. 2019(09): 139-141+144 .
49.	王丽，康飞. 基于樽海鞘群优化算法的裂缝图像分割. 电脑知识与技术. 2019(25): 223-224 .