A Multi-Dimensional Scenario-Based Evaluation Method for Deep Learning Side-Channel Analysis Using a Multi-Attribute Decision Model
-
摘要: 针对深度学习侧信道分析(DL-SCA)模型评估中存在的维度单一、公平性不足以及与工程场景脱节的问题，该文提出一种基于系统工程的多维度场景化评估框架。该框架首先构建覆盖攻击效能、资源开销与环境适应性的层次化评估指标体系；其次，设计一种CRITIC-AHP混合多属性决策机制，融合数据驱动的客观赋权与场景导向的主观权重，实现评估与不同应用需求的精准匹配；在此基础上，定义多维度攻击性能指标，融合多维度信息，生成直观、可比的综合评分，为模型优选提供统一量化依据。基于ASCAD数据集的实验表明，所提框架在资源受限、高性能、高噪声及实时等典型场景下均能有效区分模型优势，如在资源受限场景中CNN综合评分最高(0.723)，在高噪声环境下CNN-LSTM表现最优(0.863)，显著提升了模型选型的科学性与可解释性。Abstract:
Objective Deep Learning Side-Channel Analysis (DL-SCA) has substantially improved the effectiveness of attacks against protected cryptographic implementations. However, the transition of DL-SCA models from research to practical deployment is limited by the lack of systematic, fair, and scenario-specific evaluation methods. Existing evaluations mainly rely on Guessing Entropy (GE) and Success Rate (SR), while overlooking practical factors such as resource overhead and environmental adaptability. Moreover, inconsistent hyperparameter optimization leads to unfair model comparisons and provides limited quantitative guidance for model selection under different deployment constraints, including resource-constrained devices, high-noise environments, and real-time applications. This paper proposes a systems engineering-based evaluation framework that enables comprehensive, quantitative, and scenario-specific assessment of DL-SCA models. Methods A multi-dimensional, scenario-based evaluation framework is developed using systems engineering principles. First, a hierarchical evaluation index system is established, comprising three criteria—attack effectiveness, resource overhead, and environmental adaptability—and six evaluation metrics: GE, SR, training time (TC), peak memory consumption (MC), model complexity (MoC), and noise robustness (Rob). Second, a standardized evaluation process based on the V-model is designed to ensure fair comparison. Each candidate model, including a Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN), and CNN-LSTM hybrid model, undergoes independent hyperparameter optimization using grid search before multi-dimensional performance evaluation. Third, a hybrid Criteria Importance Through Intercriteria Correlation-Analytic Hierarchy Process (CRITIC-AHP) Multi-Attribute Decision-Making (MADM) framework is developed. The CRITIC method derives objective weights from the statistical characteristics of the evaluation data, whereas the AHP method incorporates scenario-specific preferences through pairwise comparison matrices. The objective and subjective weights are fused to generate scenario-specific weights. Finally, a Multi-dimensional Attack Performance Metric (MAPM) is defined as the weighted sum of normalized evaluation metrics using the fused weights, providing a composite score for each model under a specific deployment scenario. Results and Discussions The proposed framework is validated using the ASCAD fixed-key dataset. After independent hyperparameter optimization, the three model architectures are evaluated using all six metrics. The CRITIC method produces the objective weight vector W critic = [0.17, 0.19, 0.15, 0.21, 0.14, 0.14]. Four representative deployment scenarios—Resource-Constrained, High-Performance, High-Noise, and Real-Time—are then defined, and the corresponding AHP preference weights are fused with the objective weights to generate the final scenario-specific weights. For example, MC receives the highest weight (0.52) in the Resource-Constrained scenario, whereas Rob dominates the High-Noise scenario with a weight of 0.57. The resulting MAPM scores ( Table 9 ,Fig. 9 , andFig. 10 ) clearly differentiate the strengths of the evaluated models and demonstrate the scenario-specific decision capability of the proposed framework. CNN achieves the highest score in the High-Performance scenario (0.894), MLP ranks first in the Real-Time scenario (0.758) because of its shortest training time, and the CNN-LSTM hybrid model performs best in the High-Noise scenario (0.863) because of its superior noise robustness despite higher resource overhead. These results demonstrate that no single model is optimal across all deployment scenarios and that MAPM provides a clear and quantitative basis for model selection under specific deployment constraints.Conclusions This paper proposes a systems engineering-based, multi-dimensional evaluation framework to address the major limitations of current DL-SCA model assessment. By integrating a hierarchical evaluation index system, a standardized V-model evaluation process, and a hybrid CRITIC-AHP Multi-Attribute Decision-Making (MADM) framework, the proposed method quantitatively balances the trade-offs among attack effectiveness, resource overhead, and environmental adaptability. Experimental results obtained using the ASCAD benchmark demonstrate that the framework provides clear, quantitative, and scenario-specific guidance for model selection. The proposed Multi-dimensional Attack Performance Metric (MAPM) provides a practical decision basis for selecting DL-SCA models under diverse deployment constraints, narrowing the gap between academic attack development and practical model deployment. Future work will extend the framework to additional model architectures and datasets, improve evaluation automation, and validate its effectiveness in practical deployment environments. -
表 1 基于层次化分解的评估指标体系
层次 名称 要素 说明 目标层 模型优选 选择最优DL-SCA模型 评估的最终目标 准则层 攻击效能 恢复密钥的核心能力 功能性准则 资源开销 计算与存储成本 非功能性准则 环境适应性 噪声下的稳定性 环境性准则 指标层 猜测熵 (GE) 负向指标 衡量攻击效率 成功率 (SR) 正向指标 衡量即时有效性 时间复杂度 (TC) 负向指标 训练时间成本 空间复杂度 (MC) 负向指标 峰值内存成本 模型复杂度 (MoC) 负向指标 参数量(存储成本) 噪声鲁棒性 (Rob) 正向指标 抗干扰能力 方案层 候选模型 MLP, CNN, CNN-LSTM 待评估的实体 
下载: 导出CSV
表 2 实验环境配置
配置项 参数 CPU Intel Core i7-11700K GPU NVIDIA GeForce RTX 4090 (24 GB) 内存 128 GB 操作系统 Windows11 深度学习框架 TensorFlow 2.10.0,Keras 2.10.0 CUDA/cuDNN 11.2/8.1.0
下载: 导出CSV
表 3 超参寻优网络配置表
Hyperparameters MLP CNN CNNLSTM FC Layers 5 2 - Neurons 200 [512,256] - Filters - [64,128,256,512] 4 Kernel size - 3 50 Conv layers - 4 1 Lstm units - - 128 Batch size 100 300 200 Activation ReLU ReLU ReLU Learning rate 1e-5 1e-4 1e-5 Epoch 300 200 500 Optimizer RMSprop RMSprop RMSprop
下载: 导出CSV
表 4 攻击效能指标对比(测试集：N=10,000条轨迹)
迹数量 CNN_GE CNN_SR MLP_GE MLP_SR CNN-LSTM_GE CNN-LSTM_SR 100 76.00 0 85.0 0 90.0 0 300 12.00 0.550 45.0 0.10 70.0 0.05 500 1.50 0.880 25.0 0.45 55.0 0.15 1000 0.20 0.980 12.0 0.75 40.0 0.30 1500 0.05 0.995 5.5 0.99 28.0 0.50 2000 0 1.000 0.3 1.00 8.0 0.90 3000 0 1.000 0 1.00 3.0 0.97 4000 0 1.000 0 1.00 0 1.00
下载: 导出CSV
表 5 资源开销指标对比
模型 参数量(MoC/万) 训练时间(TC/s) 峰值内存(MC/MB) MLP 35 568 1950 CNN 425 2850 520 CNN-LSTM 78 1450 3050
下载: 导出CSV
表 6 鲁棒性(Rob)指标对比
噪声水平 MLPSR MLPGE CNNSR CNNGE CNN-LSTMSR CNN-LSTMGE 0.0 0.992 0.08 0.784 10.25 0.865 6.45 0.1 0.980 0.15 0.770 12.50 0.860 7.20 0.2 0.960 0.30 0.750 15.80 0.855 8.50 0.3 0.930 0.65 0.720 20.10 0.850 10.20 0.4 0.890 1.20 0.680 25.50 0.845 12.80 0.5 0.840 2.10 0.630 32.00 0.840 16.00 0.6 0.780 3.50 0.570 40.20 0.835 20.50 0.7 0.710 5.80 0.500 50.10 0.830 25.80
下载: 导出CSV
表 7 高噪声场景AHP判断矩阵及一致性检验
指标 GE SR TC MC MoC Rob 权重 GE 1 1 3 3 3 1/4 0.15 SR 1 1 3 3 3 1/4 0.15 TC 1/3 1/3 1 1 1 1/5 0.06 MC 1/3 1/3 1 1 1 1/5 0.06 MoC 1/3 1/3 1 1 1 1/5 0.06 Rob 4 4 5 5 5 1 0.52
下载: 导出CSV
表 8 各场景最终权重分配
评估指标 资源受限场景 高性能场景 高噪声场景 实时场景 猜测熵(GE) 0.04 0.37 0.06 0.09 成功率(SR) 0.06 0.41 0.04 0.11 训练时间(TC) 0.11 0.05 0.12 0.38 内存占用(MC) 0.52 0.06 0.13 0.29 模型复杂度(MoC) 0.12 0.04 0.08 0.08 鲁棒性(Rob) 0.15 0.07 0.57 0.05
下载: 导出CSV
表 9 多维度场景化综合评估(MAPM)结果与排名
模型 资源受限场景 排名 高性能场景 排名 高噪声场景 排名 实时场景 排名 CNN 0.723 1 0.894 1 0.382 3 0.501 2 MLP 0.608 2 0.832 2 0.785 2 0.758 1 CNN-LSTM 0.289 3 0.214 3 0.863 1 0.324 3
下载: 导出CSV
表 10 场景-指标约束映射表
场景 典型部署环境 核心约束 对应指标 数据来源 资源受限 物联网边缘节点、智能卡、低功耗MCU 存储与算力严格受限 峰值内存(MC)、参数量(MoC) 来自训练过程测量 高性能 GPU服务器、云端计算平台 资源充裕，唯攻击效能论 猜测熵(GE)、成功率(SR) 来自测试集攻击实验 高噪声 工业现场、电磁泄露远距离/非侵入攻击 环境信噪比低，干扰强烈 噪声鲁棒性(Rob) 来自加噪测试 实时攻击 在线攻击系统、支付终端、车载/物联网IDS 训练或推理时延严格受限 训练时间(TC) 来自训练过程测量
下载: 导出CSV
-
[1] Kocher P C. Timing Attacks on Implementations of Diffie-Hellman, RSA, DSS, and Other Systems[C]. Advances in Cryptology - CRYPTO 96, 16th Annual International Cryptology Conference, Santa Barbara, California, USA, 1996: 104–113. doi: 10.1007/3-540-68697-5_9. [2] 郑震, 严迎建, 刘燕江. 侧信道能量信息测试向量泄漏评估技术[J]. 电子与信息学报, 2023, 45(9): 3109–3117. doi: 10.11999/JEIT230295.ZHENG Zhen, YAN Yingjian, and LIU Yanjiang. Test vector leakage assessment technique of side-channel power information[J]. Journal of Electronics Information Technology, 2023, 45(9): 3109–3117. doi: 10.11999/JEIT230295. [3] Brier E, Clavier C, and Olivier F. Correlation Power Analysis with a Leakage Model[C]. Cryptographic Hardware and Embedded Systems - CHES 2004, 6th International Workshop, Cambridge, MA, USA, 2004: 16–29. doi: 10.1007/978-3-540-28632-5_2. [4] 胡伟, 袁超绚, 郑健, 等. 一种针对格基后量子密码的能量侧信道分析框架[J]. 电子与信息学报, 2023, 45(9): 3210–3217. doi: 10.11999/JEIT230267.HU Wei, YUAN Chaoxuan, ZHENG Jian, et al. A power side-channel attack framework for lattice-based post quantum cryptography[J]. Journal of Electronics & Information Technology, 2023, 45(9): 3210–3217. doi: 10.11999/JEIT230267. [5] Maghrebi H, Portigliatti T, and Prouff E. Breaking Cryptographic Implementations Using Deep Learning Techniques[C]. Security, Privacy, and Applied Cryptography Engineering - 6th International Conference, SPACE 2016, Hyderabad, India, 2016: 3–26. doi: 10.1007/978-3-319-49445-6_1. [6] Hettwer B, Gehrer S, and Güneysu T. Deep Neural Network Attribution Methods for Leakage Analysis and Symmetric Key Recovery[C]. Selected Areas in Cryptography – SAC 2019, Waterloo, Canada, 2019: 645–666. doi: 10.1007/978-3-030-38471-5_26. [7] PROUFF E, STRULLU R, BENADJILA R, et al. Study of deep learning techniques for side-channel analysis and introduction to ASCAD database[J]. Journal of Cryptographic Engineering, 2020, 10(2): 163–188 doi: 10.1007/s13389-020-00220-8. [8] 严迎建, 常雅静, 朱春生, 刘燕江. 基于循环密文的格密码模板攻击方法[J]. 电子与信息学报, 2023, 45(12): 4530–4538. doi: 10.11999/JEIT221164.YAN Yingjian, CHANG Yajing, ZHU Chunsheng, et al. A lattice cipher template attack method based on recurrent cryptography[J]. Journal of Electronics & Information Technology, 2023, 45(12): 4530–4538. doi: 10.11999/JEIT221164. [9] Gohr A. Improving Attacks on Round-Reduced Speck32/64 Using Deep Learning[C]. Advances in Cryptology - CRYPTO 2019, Santa Barbara, California, USA, 2019: 150–179. doi: 10.1007/978-3-030-26951-7_6. [10] WOUTERS L, ARRIBAS V, GIERLICHS B, et al. Revisiting a methodology for efficient CNN architectures in profiling attacks[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2020, 2020(3): 147–168. doi: 10.13154/tches.v2020.i3.147-168. [11] AHMED A A, HASAN M K, MEMON I, et al. Secure AI for 6G mobile devices: Deep learning optimization against side-channel attacks[J]. IEEE Transactions on Consumer Electronics, 2024, 70(1): 3951–3959 doi: 10.1109/TCE.2024.3372018. [12] ZAID G, BOSSUET L, HABRARD A, et al. Methodology for efficient CNN architectures in profiling attacks[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2020, 2020(1): 1–36 doi: 10.13154/tches.v2020.i1.1-36. [13] BENADJILA R, PROUFF E, STRULLU R, et al. Study of deep learning techniques for side-channel analysis and introduction to ASCAD database[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2018, 2018(3): 1–35. doi: 10.46586/tches.v2018.i3.1-35. [14] Perin G, Wu L, and Picek S. Gambling for Success: The Lottery Ticket Hypothesis in Deep Learning-Based Side-Channel Analysis[C]. Constructive Side-Channel Analysis and Secure Design – 13th International Workshop, COSADE 2022, Leuven, Belgium, 2022: 217-241. doi: 10.1007/978-3-030-97087-1_9. [15] RIJSDIJK J, WU Lichao, PERIN G, et al. Reinforcement learning for hyperparameter tuning in deep learning-based side-channel analysis[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2021, 2021(3): 677–707. doi: 10.46586/tches.v2021.i3.677-707. [16] Wu L, Perin G, and Picek S. Weakly Profiling Side-channel Analysis[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2024, 2024(3): 707–730 doi: 10.46586/tches.v2024.i3.707-730. [17] ZHU Junfan, LU Jiqiang. Leading Degree: A Metric for Model Performance Evaluation and Hyperparameter Tuning in Deep Learning-Based Side-Channel Analysis[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2025, 2025(2): 333–361 doi: 10.46586/tches.v2025.i2.333-361. [18] WANG J N, OUYANG Q X, WANG H Y. A systematic evaluation of deep-learning side-channel attacks: Performance and cost[C]. The 2026 5th International Conference on Cryptography, Network Security and Communication Technology (CNSCT 2026). New York: ACM, 2026. doi: 10.1145/3802927.3802950. [19] PERIN G, WU L, PICEK S. Exploring feature selection scenarios for deep learning-based side-channel analysis[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2022, 2022(4): 828–861 doi: 10.46586/tches.v2022.i4.828-861. [20] 耿涛, 张安, 郝兴国. 基于组合赋权TOPSIS法的空战多目标威胁评估[J]. 火力与指挥控制, 2011, 36(3): 16–19 doi: CNKI:SUN:HLYZ.0.2011-03-005.GENG Tao, ZHANG An, and HAO Xingguo. Multi-target threat assessment in air combat based on combination determining weights TOPSIS[J]. Fire Control & Command Control, 2011, 36(3): 16–19 doi: CNKI:SUN:HLYZ.0.2011-03-005. [21] Wu L, Perin G, and Picek S. On the Evaluation of Deep Learning-Based Side-Channel Analysis[C]. Constructive Side-Channel Analysis and Secure Design - 13th International Workshop, COSADE 2022, Leuven, Belgium, 2022: 49-71. doi: 10.1007/978-3-030-99766-3_3. [22] 徐杨, 李锴彬, 何星星. 融合卷积块注意力机制与三元组度量学习的深度侧信道攻击方法[J]. 电子与信息学报, 2026, 优先出版. doi: 10.11999/JEIT260140.XU Yang, LI Kaibin, HE Xingxing. Deep side-channel attack method integrating convolutional block attention mechanism and triplet metric learning[J]. Journal of Electronics & Information Technology, 2026, in press. doi: 10.11999/JEIT260140. [23] Wu L, Perin G, and Picek S. I Choose You: Automated Hyperparameter Tuning for Deep Learning-based Side-Channel Analysis[J]. IEEE Transactions on Emerging Topics in Computing, 2024, 12(2): 546–557 doi: 10.1109/TETC.2022.3218372. [24] ITO A, UENO R, HOMMA N. Perceived information revisited II: Information-theoretical analysis of deep-learning based side-channel attacks[J]. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2025, 2025(1): 450–474 doi: 10.46586/tches.v2025.i1.450-474. [25] LIU Weifeng, LI Wenchang, CAO Xiaodong, et al. Full-element analysis of side-channel leakage dataset on symmetric cryptographic advanced encryption standard[J]. Symmetry, 2025, 17(5): 769. doi: 10.3390/sym17050769. -
下载:
图(9) / 表(10)
计量
- 文章访问数: 384
- HTML全文浏览量: 239
- PDF下载量: 29
- 被引次数: 0
