A Multi-Dimensional Scenario-Based Evaluation Method for Deep Learning Side-Channel Analysis Using a Multi-Attribute Decision Model

GU Zepeng; CHEN Lin; CAI Juesong; YAN Yingjian

doi:10.11999/JEIT260198

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 >

GU Zepeng, CHEN Lin, CAI Juesong, YAN Yingjian. A Multi-Dimensional Scenario-Based Evaluation Method for Deep Learning Side-Channel Analysis Using a Multi-Attribute Decision Model[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260198

Citation:

GU Zepeng, CHEN Lin, CAI Juesong, YAN Yingjian. A Multi-Dimensional Scenario-Based Evaluation Method for Deep Learning Side-Channel Analysis Using a Multi-Attribute Decision Model[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260198

Citation:

GU Zepeng, CHEN Lin, CAI Juesong, YAN Yingjian. A Multi-Dimensional Scenario-Based Evaluation Method for Deep Learning Side-Channel Analysis Using a Multi-Attribute Decision Model[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260198

PDF( 3082 KB)

A Multi-Dimensional Scenario-Based Evaluation Method for Deep Learning Side-Channel Analysis Using a Multi-Attribute Decision Model

doi: 10.11999/JEIT260198 cstr: 32379.14.JEIT260198

College of Cryptography Engineering, Information Engineering University, Zhengzhou 450001, China

Received Date: 2026-02-14
Accepted Date: 2026-06-24
Rev Recd Date: 2026-06-24

Available Online: 2026-07-04

Abstract

Abstract

Objective The application of deep learning has significantly advanced side-channel analysis (DL-SCA), enabling attacks against protected implementations. However, transitioning DL-SCA models from research to practical deployment is hindered by the lack of systematic, fair, and scenario-aware evaluation methodologies. Current evaluations predominantly rely on one-dimensional metrics like Guessing Entropy (GE) and Success Rate (SR), neglecting critical practical dimensions such as resource consumption and environmental robustness. Furthermore, comparisons are often unfair due to inconsistent hyperparameter optimization, and they fail to provide quantifiable guidance for model selection tailored to diverse real-world constraints (e.g., resource-limited devices, high-noise environments, or real-time requirements). This paper aims to address these gaps by proposing a comprehensive, systems engineering-based evaluation framework that enables holistic, quantifiable, and scenario-adaptive assessment of DL-SCA models. Methods A multi-dimensional, scenario-based evaluation framework is constructed based on systems engineering principles. First, a hierarchical evaluation index system is established, encompassing three criteria (attack efficacy, resource overhead, and environmental adaptability) and six specific metrics (GE, SR, training time TC, peak memory consumption MC, model complexity MoC, and noise robustness Rob). Second, a standardized evaluation process following the "V-model" is designed to ensure fairness. This process mandates independent hyperparameter optimization for each candidate model (MLP, CNN, CNN-LSTM) using grid search before comprehensive multi-dimensional data collection. Third, the core of the framework is a hybrid CRITIC-AHP (Criteria Importance Through Intercriteria Correlation - Analytic Hierarchy Process) Multi-Attribute Decision Making (MADM) engine. The CRITIC method derives objective weights from the statistical characteristics (contrast intensity and conflict) of the measured data matrix. The AHP method incorporates subjective, scenario-specific preferences (e.g., prioritizing low memory or high robustness) through pairwise comparison matrices. These weights are fused to generate final scenario-adapted weights. Finally, a Multi-dimensional Attack Performance Metric (MAPM) is defined as the linear weighted sum of normalized metric values using the fused weights, providing a single, comparable score for each model under a given scenario. Results and Discussions The framework is rigorously validated using the standard ASCAD fixed-key dataset. After independent optimization, the three model architectures are evaluated across all six metrics. The CRITIC method yields an objective base weight vector: W_critic = [0.17, 0.19, 0.15, 0.21, 0.14, 0.14]. For four predefined scenarios (Resource-Constrained, High-Performance, High-Noise, Real-Time), specific AHP judgments are made and fused with the objective weights to produce the final adapted weights (Table 8). For instance, in the Resource-Constrained scenario, memory consumption (MC) receives the highest weight (0.52), while in the High-Noise scenario, robustness (Rob) is dominant (0.57). The calculated MAPM scores (Table 9, Fig.9, Fig.10) clearly quantify the differentiated advantages of each model and demonstrate the framework's scenario-aware decision capability: CNN achieves the highest score in High-Performance scenarios (0.894), MLP excels in Real-Time scenarios (0.758) due to its lowest training time, and CNN-LSTM performs best in High-Noise scenarios (0.863) owing to its superior robustness, despite its higher resource cost. These results effectively prove that there is no universally "best" model and that the proposed MAPM provides a clear, quantitative basis for model selection under specific engineering constraints. Conclusions This paper proposes a novel systems engineering-based, multi-dimensional evaluation framework to address the key limitations in current DL-SCA model assessment. By integrating a hierarchical index system, a fair V-model process, and a hybrid CRITIC-AHP MADM engine, the framework successfully quantifies and balances the trade-offs between attack efficacy, resource cost, and environmental adaptability. The experimental results on the ASCAD benchmark demonstrate its practical utility in generating clear, quantifiable, and scenario-aware model selection guidelines. The proposed MAPM offers a direct decision basis for engineers facing diverse deployment contexts, bridging the gap between academic attack construction and practical model deployment in DL-SCA. Future work may involve extending the evaluation to more model architectures and datasets, enhancing the automation of the framework, and validating it in real-world deployment scenarios.
- Side-channel analysis (SCA),
- Deep learning,
- Model evaluation,
- Systems engineering,
- Multi-dimensional scenario-based evaluation

FullText(HTML)

References(25)

References

[1]	KOCHER P C. Timing attacks on implementations of Diffie-Hellman, RSA, DSS, and other systems[C]. Advances in Cryptology - CRYPTO’96, 16th Annual International Cryptology Conference, Santa Barbara, USA, 1996: 104–113. doi: 10.1007/3-540-68697-5_9.
[2]	KOCHER P, JAFFE J, and JUN B. Differential power analysis[C]. Advances in Cryptology - CRYPTO’99, 9th Annual International Cryptology Conference, Santa Barbara, USA, 1999: 388–397. doi: 10.1007/3-540-48405-1_25.
[3]	BRIER E, CLAVIER C, and OLIVIER F. Correlation power analysis with a leakage model[C]. Cryptographic Hardware and Embedded Systems - CHES 2004, 6th International Workshop, Cambridge, USA, 2004: 16–29. doi: 10.1007/978-3-540-28632-5_2.
[4]	ZHANG Fan, DONG Xiaofei, YANG Bolin, et al. A systematic evaluation of wavelet-based attack framework on random delay countermeasures[J]. IEEE Transactions on Information Forensics and Security, 2020, 15: 1407–1422. doi: 10.1109/TIFS.2019.2941774.
[5]	MAGHREBI H, PORTIGLIATTI T, and PROUFF E. Breaking cryptographic implementations using deep learning techniques[C]. Security, Privacy, and Applied Cryptography Engineering, 6th International Conference, SPACE 2016, Hyderabad, India, 2016: 3–26. doi: 10.1007/978-3-319-49445-6_1.
[6]	HETTWER B, GEHRER S, and GUNEYSU T. Deep neural network based cryptanalysis of lightweight block ciphers[J]. IACR Transactions on Symmetric Cryptology, 2020, 2020(3): 49–78. doi: 10.46586/tosc.v2020.i3.49-78.
[7]	BENADJILA R, PROUFF E, STRULLU R, et al. Deep learning for side-channel analysis and introduction to ASCAD database[J]. Journal of Cryptographic Engineering, 2020, 10(2): 163–188. doi: 10.1007/s13389-019-00220-8.
[8]	MARTINAZZI S, ZANKL A, SCHILLING M, et al. A systematic review of deep learning for side-channel analysis: Challenges and opportunities[J]. IEEE Transactions on Information Forensics and Security, 2024, 19: 1058–1073. doi: 10.1109/TIFS.2023.3321123.
[9]	GOHR A. Improving attacks on round-reduced speck32/64 using deep learning[C]. Advances in Cryptology - CRYPTO 2019, 39th Annual International Cryptology Conference, Santa Barbara, USA, 2019: 150–179. doi: 10.1007/978-3-030-26951-7_6.
[10]	WOUTERS L, ARRIBAS V, GIERLICHS B, et al. Revisiting a methodology for efficient CNN architectures in profiling attacks[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2020, 2020(3): 147–168. doi: 10.13154/tches.v2020.i3.147-168.
[11]	PARK D, LEE K, and KIM H. CASTLE: A context-aware strategy for tunable evaluation of deep learning SCA models[J]. IEEE Transactions on Dependable and Secure Computing, 2024, 21(3): 2145–2159. doi: 10.1109/TDSC.2023.3258246.
[12]	ZAID G, BOSSUET L, HARBRECHT H, et al. Towards efficient and scalable side-channel attacks modeling using convolutional neural networks[C]. 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), Grenoble, France, 2020: 133–138. (查阅网上资料, 未找到本条文献信息, 请确认).
[13]	BENADJILA R, PROUFF E, STRULLU R, et al. Study of deep learning techniques for side-channel analysis and introduction to ASCAD database[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2018, 2018(3): 1–35. doi: 10.46586/tches.v2018.i3.1-35.
[14]	WANG Z, LIU Y, SONG N, et al. Robustness assessment of deep learning-based side-channel analysis against adversarial trace perturbations[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2024, 43(5): 789–802. doi: 10.1109/TCAD.2023.3334567.
[15]	RIJSDIJK J, WU Lichao, PERIN G, et al. Reinforcement learning for hyperparameter tuning in deep learning-based side-channel analysis[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2021, 2021(3): 677–707. doi: 10.46586/tches.v2021.i3.677-707.
[16]	WU L, PERIN G, and PICEK S. The (un)fairness of deep learning-based side-channel analysis: A large-scale benchmarking study[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2024, 2024(2): 1–30. doi: 10.46586/tches.v2024.i2.1-30.
[17]	ZHANG J, WANG H, LIU Z, et al. Evaluating robustness of deep learning-based side-channel attacks against adversarial traces[C]. 2023 International Conference on Cyber Security and Protection of Digital Services (Cyber Security), Oxford, UK, 2023: 1–8. (查阅网上资料, 未找到本条文献信息, 请确认).
[18]	CHEN L, WANG Y, LIU J, et al. A lightweight CNN architecture for side-channel analysis on embedded devices[J]. IEEE Transactions on Circuits and Systems II: Express Briefs, 2024, 71(2): 456–460. doi: 10.1109/TCSII.2023.3329876.
[19]	KUMAR A, ZHOU Y, and BHATTACHARYA S. On the trade-offs between model complexity and attack efficiency in deep learning-based SCA[C]. Proceedings of the 2023 ACM Workshop on Attacks and Solutions in Hardware Security (ASHES’23), New York, USA, 2023: 45–52. (查阅网上资料, 未找到本条文献信息, 请确认).
[20]	SÁNCHEZ P, ROJAS E, and ROY D B. MESA: A multi-objective evaluation framework for security applications using systematic weighting[J]. ACM Transactions on Privacy and Security, 2023, 26(4): 1–30. doi: 10.1145/3592612. (查阅网上资料,未找到本条文献信息,请确认).
[21]	PERIN G, WU L, and PICEK S. Exploring the trade-offs: Model accuracy vs. complexity in deep learning SCA[C]. Constructive Side-Channel Analysis and Secure Design – COSADE 2022, Milan, Italy, 2022: 189–209. (查阅网上资料, 未找到本条文献信息, 请确认).
[22]	DUBEY A and MUKHOPADHYAY D. Noise tolerance of deep learning based side channel attacks: An experimental study[J]. Journal of Cryptographic Engineering, 2023, 13(4): 431–449. doi: 10.1007/s13389-022-00311-x.
[23]	LEE K and PARK D. AutoSCA: An automated framework for fair and reproducible side-channel analysis with deep learning[J]. IEEE Access, 2024, 12: 45678–45692. doi: 10.1109/ACCESS.2024.3369876.
[24]	DUBOIS S, NAJM Z, and DANGER J L. One metric to rule them all? A critical discussion on the evaluation of deep learning-based side-channel attacks[C]. Constructive Side-Channel Analysis and Secure Design – COSADE 2023, Munich, Germany, 2023: 89–110. (查阅网上资料, 未找到本条文献信息, 请确认).
[25]	LIU Weifeng, LI Wenchang, CAO Xiaodong, et al. Full-element analysis of side-channel leakage dataset on symmetric cryptographic advanced encryption standard[J]. Symmetry, 2025, 17(5): 769. doi: 10.3390/sym17050769.