A Multi-Dimensional Scenario-Based Evaluation Method for Deep Learning Side-Channel Analysis Using a Multi-Attribute Decision Model

GU Zepeng; CHEN Lin; CAI Juesong; YAN Yingjian

doi:10.11999/JEIT260198

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

GU Zepeng, CHEN Lin, CAI Juesong, YAN Yingjian. A Multi-Dimensional Scenario-Based Evaluation Method for Deep Learning Side-Channel Analysis Using a Multi-Attribute Decision Model[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260198

Citation:

GU Zepeng, CHEN Lin, CAI Juesong, YAN Yingjian. A Multi-Dimensional Scenario-Based Evaluation Method for Deep Learning Side-Channel Analysis Using a Multi-Attribute Decision Model[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260198

Citation:

GU Zepeng, CHEN Lin, CAI Juesong, YAN Yingjian. A Multi-Dimensional Scenario-Based Evaluation Method for Deep Learning Side-Channel Analysis Using a Multi-Attribute Decision Model[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260198

PDF( 4471 KB)

A Multi-Dimensional Scenario-Based Evaluation Method for Deep Learning Side-Channel Analysis Using a Multi-Attribute Decision Model

doi: 10.11999/JEIT260198 cstr: 32379.14.JEIT260198

College of Cryptography Engineering, Information Engineering University, Zhengzhou 450001, China

Received Date: 2026-02-14
Accepted Date: 2026-06-24
Rev Recd Date: 2026-06-24

Available Online: 2026-07-04

Abstract

Abstract

Objective Deep Learning Side-Channel Analysis (DL-SCA) has substantially improved the effectiveness of attacks against protected cryptographic implementations. However, the transition of DL-SCA models from research to practical deployment is limited by the lack of systematic, fair, and scenario-specific evaluation methods. Existing evaluations mainly rely on Guessing Entropy (GE) and Success Rate (SR), while overlooking practical factors such as resource overhead and environmental adaptability. Moreover, inconsistent hyperparameter optimization leads to unfair model comparisons and provides limited quantitative guidance for model selection under different deployment constraints, including resource-constrained devices, high-noise environments, and real-time applications. This paper proposes a systems engineering-based evaluation framework that enables comprehensive, quantitative, and scenario-specific assessment of DL-SCA models. Methods A multi-dimensional, scenario-based evaluation framework is developed using systems engineering principles. First, a hierarchical evaluation index system is established, comprising three criteria—attack effectiveness, resource overhead, and environmental adaptability—and six evaluation metrics: GE, SR, training time (TC), peak memory consumption (MC), model complexity (MoC), and noise robustness (Rob). Second, a standardized evaluation process based on the V-model is designed to ensure fair comparison. Each candidate model, including a Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN), and CNN-LSTM hybrid model, undergoes independent hyperparameter optimization using grid search before multi-dimensional performance evaluation. Third, a hybrid Criteria Importance Through Intercriteria Correlation-Analytic Hierarchy Process (CRITIC-AHP) Multi-Attribute Decision-Making (MADM) framework is developed. The CRITIC method derives objective weights from the statistical characteristics of the evaluation data, whereas the AHP method incorporates scenario-specific preferences through pairwise comparison matrices. The objective and subjective weights are fused to generate scenario-specific weights. Finally, a Multi-dimensional Attack Performance Metric (MAPM) is defined as the weighted sum of normalized evaluation metrics using the fused weights, providing a composite score for each model under a specific deployment scenario. Results and Discussions The proposed framework is validated using the ASCAD fixed-key dataset. After independent hyperparameter optimization, the three model architectures are evaluated using all six metrics. The CRITIC method produces the objective weight vector W _critic = [0.17, 0.19, 0.15, 0.21, 0.14, 0.14]. Four representative deployment scenarios—Resource-Constrained, High-Performance, High-Noise, and Real-Time—are then defined, and the corresponding AHP preference weights are fused with the objective weights to generate the final scenario-specific weights. For example, MC receives the highest weight (0.52) in the Resource-Constrained scenario, whereas Rob dominates the High-Noise scenario with a weight of 0.57. The resulting MAPM scores (Table 9, Fig. 9, and Fig. 10) clearly differentiate the strengths of the evaluated models and demonstrate the scenario-specific decision capability of the proposed framework. CNN achieves the highest score in the High-Performance scenario (0.894), MLP ranks first in the Real-Time scenario (0.758) because of its shortest training time, and the CNN-LSTM hybrid model performs best in the High-Noise scenario (0.863) because of its superior noise robustness despite higher resource overhead. These results demonstrate that no single model is optimal across all deployment scenarios and that MAPM provides a clear and quantitative basis for model selection under specific deployment constraints. Conclusions This paper proposes a systems engineering-based, multi-dimensional evaluation framework to address the major limitations of current DL-SCA model assessment. By integrating a hierarchical evaluation index system, a standardized V-model evaluation process, and a hybrid CRITIC-AHP Multi-Attribute Decision-Making (MADM) framework, the proposed method quantitatively balances the trade-offs among attack effectiveness, resource overhead, and environmental adaptability. Experimental results obtained using the ASCAD benchmark demonstrate that the framework provides clear, quantitative, and scenario-specific guidance for model selection. The proposed Multi-dimensional Attack Performance Metric (MAPM) provides a practical decision basis for selecting DL-SCA models under diverse deployment constraints, narrowing the gap between academic attack development and practical model deployment. Future work will extend the framework to additional model architectures and datasets, improve evaluation automation, and validate its effectiveness in practical deployment environments.
- Side-Channel Analysis (SCA),
- Deep learning,
- Model evaluation,
- Systems engineering,
- Multi-dimensional scenario-based evaluation

FullText(HTML)

References(25)

References

[1]	Kocher P C. Timing Attacks on Implementations of Diffie-Hellman, RSA, DSS, and Other Systems[C]. Advances in Cryptology - CRYPTO 96, 16th Annual International Cryptology Conference, Santa Barbara, California, USA, 1996: 104–113. doi: 10.1007/3-540-68697-5_9.
[2]	郑震, 严迎建, 刘燕江. 侧信道能量信息测试向量泄漏评估技术[J]. 电子与信息学报, 2023, 45(9): 3109–3117. doi: 10.11999/JEIT230295. ZHENG Zhen, YAN Yingjian, and LIU Yanjiang. Test vector leakage assessment technique of side-channel power information[J]. Journal of Electronics Information Technology, 2023, 45(9): 3109–3117. doi: 10.11999/JEIT230295.
[3]	Brier E, Clavier C, and Olivier F. Correlation Power Analysis with a Leakage Model[C]. Cryptographic Hardware and Embedded Systems - CHES 2004, 6th International Workshop, Cambridge, MA, USA, 2004: 16–29. doi: 10.1007/978-3-540-28632-5_2.
[4]	胡伟, 袁超绚, 郑健, 等. 一种针对格基后量子密码的能量侧信道分析框架[J]. 电子与信息学报, 2023, 45(9): 3210–3217. doi: 10.11999/JEIT230267. HU Wei, YUAN Chaoxuan, ZHENG Jian, et al. A power side-channel attack framework for lattice-based post quantum cryptography[J]. Journal of Electronics & Information Technology, 2023, 45(9): 3210–3217. doi: 10.11999/JEIT230267.
[5]	Maghrebi H, Portigliatti T, and Prouff E. Breaking Cryptographic Implementations Using Deep Learning Techniques[C]. Security, Privacy, and Applied Cryptography Engineering - 6th International Conference, SPACE 2016, Hyderabad, India, 2016: 3–26. doi: 10.1007/978-3-319-49445-6_1.
[6]	Hettwer B, Gehrer S, and Güneysu T. Deep Neural Network Attribution Methods for Leakage Analysis and Symmetric Key Recovery[C]. Selected Areas in Cryptography – SAC 2019, Waterloo, Canada, 2019: 645–666. doi: 10.1007/978-3-030-38471-5_26.
[7]	PROUFF E, STRULLU R, BENADJILA R, et al. Study of deep learning techniques for side-channel analysis and introduction to ASCAD database[J]. Journal of Cryptographic Engineering, 2020, 10(2): 163–188 doi: 10.1007/s13389-020-00220-8.
[8]	严迎建, 常雅静, 朱春生, 刘燕江. 基于循环密文的格密码模板攻击方法[J]. 电子与信息学报, 2023, 45(12): 4530–4538. doi: 10.11999/JEIT221164. YAN Yingjian, CHANG Yajing, ZHU Chunsheng, et al. A lattice cipher template attack method based on recurrent cryptography[J]. Journal of Electronics & Information Technology, 2023, 45(12): 4530–4538. doi: 10.11999/JEIT221164.
[9]	Gohr A. Improving Attacks on Round-Reduced Speck32/64 Using Deep Learning[C]. Advances in Cryptology - CRYPTO 2019, Santa Barbara, California, USA, 2019: 150–179. doi: 10.1007/978-3-030-26951-7_6.
[10]	WOUTERS L, ARRIBAS V, GIERLICHS B, et al. Revisiting a methodology for efficient CNN architectures in profiling attacks[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2020, 2020(3): 147–168. doi: 10.13154/tches.v2020.i3.147-168.
[11]	AHMED A A, HASAN M K, MEMON I, et al. Secure AI for 6G mobile devices: Deep learning optimization against side-channel attacks[J]. IEEE Transactions on Consumer Electronics, 2024, 70(1): 3951–3959 doi: 10.1109/TCE.2024.3372018.
[12]	ZAID G, BOSSUET L, HABRARD A, et al. Methodology for efficient CNN architectures in profiling attacks[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2020, 2020(1): 1–36 doi: 10.13154/tches.v2020.i1.1-36.
[13]	BENADJILA R, PROUFF E, STRULLU R, et al. Study of deep learning techniques for side-channel analysis and introduction to ASCAD database[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2018, 2018(3): 1–35. doi: 10.46586/tches.v2018.i3.1-35.
[14]	Perin G, Wu L, and Picek S. Gambling for Success: The Lottery Ticket Hypothesis in Deep Learning-Based Side-Channel Analysis[C]. Constructive Side-Channel Analysis and Secure Design – 13th International Workshop, COSADE 2022, Leuven, Belgium, 2022: 217-241. doi: 10.1007/978-3-030-97087-1_9.
[15]	RIJSDIJK J, WU Lichao, PERIN G, et al. Reinforcement learning for hyperparameter tuning in deep learning-based side-channel analysis[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2021, 2021(3): 677–707. doi: 10.46586/tches.v2021.i3.677-707.
[16]	Wu L, Perin G, and Picek S. Weakly Profiling Side-channel Analysis[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2024, 2024(3): 707–730 doi: 10.46586/tches.v2024.i3.707-730.
[17]	ZHU Junfan, LU Jiqiang. Leading Degree: A Metric for Model Performance Evaluation and Hyperparameter Tuning in Deep Learning-Based Side-Channel Analysis[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2025, 2025(2): 333–361 doi: 10.46586/tches.v2025.i2.333-361.
[18]	WANG J N, OUYANG Q X, WANG H Y. A systematic evaluation of deep-learning side-channel attacks: Performance and cost[C]. The 2026 5th International Conference on Cryptography, Network Security and Communication Technology (CNSCT 2026). New York: ACM, 2026. doi: 10.1145/3802927.3802950.
[19]	PERIN G, WU L, PICEK S. Exploring feature selection scenarios for deep learning-based side-channel analysis[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2022, 2022(4): 828–861 doi: 10.46586/tches.v2022.i4.828-861.
[20]	耿涛, 张安, 郝兴国. 基于组合赋权TOPSIS法的空战多目标威胁评估[J]. 火力与指挥控制, 2011, 36(3): 16–19 doi: CNKI:SUN:HLYZ.0.2011-03-005. GENG Tao, ZHANG An, and HAO Xingguo. Multi-target threat assessment in air combat based on combination determining weights TOPSIS[J]. Fire Control & Command Control, 2011, 36(3): 16–19 doi: CNKI:SUN:HLYZ.0.2011-03-005.
[21]	Wu L, Perin G, and Picek S. On the Evaluation of Deep Learning-Based Side-Channel Analysis[C]. Constructive Side-Channel Analysis and Secure Design - 13th International Workshop, COSADE 2022, Leuven, Belgium, 2022: 49-71. doi: 10.1007/978-3-030-99766-3_3.
[22]	徐杨, 李锴彬, 何星星. 融合卷积块注意力机制与三元组度量学习的深度侧信道攻击方法[J]. 电子与信息学报, 2026, 优先出版. doi: 10.11999/JEIT260140. XU Yang, LI Kaibin, HE Xingxing. Deep side-channel attack method integrating convolutional block attention mechanism and triplet metric learning[J]. Journal of Electronics & Information Technology, 2026, in press. doi: 10.11999/JEIT260140.
[23]	Wu L, Perin G, and Picek S. I Choose You: Automated Hyperparameter Tuning for Deep Learning-based Side-Channel Analysis[J]. IEEE Transactions on Emerging Topics in Computing, 2024, 12(2): 546–557 doi: 10.1109/TETC.2022.3218372.
[24]	ITO A, UENO R, HOMMA N. Perceived information revisited II: Information-theoretical analysis of deep-learning based side-channel attacks[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2025, 2025(1): 450–474 doi: 10.46586/tches.v2025.i1.450-474.
[25]	LIU Weifeng, LI Wenchang, CAO Xiaodong, et al. Full-element analysis of side-channel leakage dataset on symmetric cryptographic advanced encryption standard[J]. Symmetry, 2025, 17(5): 769. doi: 10.3390/sym17050769.