Moving Target Defense Strategy Optimization Scheme for Cloud Native Environment Based on Deep Reinforcement Learning
-
摘要: 针对云原生环境下攻击场景的复杂性导致移动目标防御策略配置困难的问题，该文提出一种基于深度强化学习的移动目标防御策略优化方案(SmartSCR)。首先，针对云原生环境容器化、微服务化等特点，对其安全威胁及攻击者攻击路径进行分析；然后，为了定量分析云原生复杂攻击场景下移动目标防御策略的防御效率，提出微服务攻击图模型并对防御效率进行刻画。最后，将移动目标防御策略的优化问题建模为马尔可夫决策过程，并使用深度强化学习解决云原生应用规模较大时带来的状态空间爆炸问题，对最优移动目标防御配置进行求解。实验结果表明，SmartSCR能够在云原生应用规模较大时快速收敛，并实现逼近最优的防御效率。Abstract: To deal with the difficulty of configuring Moving Target Defense (MTD) strategy under complexity attack scenarios in the cloud native environment, a deep reinforcement learning based moving target defense strategy optimization scheme (SmartSCR) is proposed. First, the security threats together with the attack paths are analyzed considering the characteristics of containerization and microservice. Then, in order to evaluate the defense efficiency of moving target defense under complexity attack scenarios in the cloud native environment, the microservice attack graph model is proposed to defense quantify efficiency. Finally, the optimization of moving target defense strategy is modeled as a Markov decision process. A deep reinforcement learning based strategy is proposed to handle the state space explosion under large scale cloud native applications, thus to solve out the optimal configuration for moving target defense strategy. The experiment results show that SmartSCR can quickly converge under large scale cloud native applications, and achieve near optimal defense efficiency.
-
Key words:
- Cloud native /
- Moving Target Defense (MTD) /
- Reinforcement learning /
- Microservice
-
算法1 基于DQN的安全配置优化算法训练 输入：微服务间调用关系 输出：DQN神经网络参数$ {\theta _j} $ (1) 初始化经验复用池的容量$ D $，最小批量经验数量$ L $，网络更
新步长$ W $(2) for episode in range(STEPS): (3) 随机生成微服务防御配置$ H $ (4) 随机生成每个微服务副本数量，并模拟调度器对副本进行
调度；(5) 基于防御配置与应用状态，生成输入$ {\mathcal{S}_t} $ (6) 以$ \varepsilon $的概率随机选择一个动作$ {a_t} $，否则选择
$ {a_t} = {\max _a}Q\left( {{\mathcal{S}_t},a;\theta } \right) $(7) 基于动作$ {a_t} $修改防御配置，得出下一个状态$ {\mathcal{S}_{t + 1}} $，并基于
MAG模型计算对应的奖励$ {r_t} $(8) 在经验复用池中存储样本$ \left\langle {{\mathcal{S}_t},{a_t},{r_t},{\mathcal{S}_{t + 1}}} \right\rangle $ (9) 从经验复用池中随机抽$ L $个样本 (10) 使用式(10)和式(11)执行梯度下降 (11) 每$ W $步更新目标网络参数$ \theta $ (12) End (13) 获取最优微服务防御配置 
下载: 导出CSV
表 1 应用漏洞信息表
微服务 名称 漏洞编号 漏洞利用困难度 漏洞权重 微服务攻击困难度 A Tomcat CVE-2021-42340 0.2564 6.8 0.3518 CVE-2021-30640 0.4545 6.0 CVE-2019-0221 0.3571 67.1 B Memcached CVE-2016-8704 0.2564 8.3 0.314 CVE-2016-8705 0.2564 8.3 CVE-2016-8706 0.4545 6.8 C ImageMagick CVE-2017-14650 0.4545 6.8 0.4010 CVE-2017-14224 0.3571 8.3 D Mysql CVE-2020-11974 0.2564 7.3 0.4179 CVE-2016-6663 1.0000 4.3 CVE-2016-6662 0.2564 8.2 – Container CVE-2021-437847 0.6250 7.6 0.4483 CVE-2020-35197 0.2564 7.0
下载: 导出CSV
-
[1] 中国信息通信研究院. 云计算白皮书[R]. 中国信息通信研究院, 2021.China Academy of Information and Communications Technology. Cloud computing white paper[R]. China Academy of Information and Communications Technology, 2021. [2] ZHOU Xiang, PENG Xin, XIE Tao, et al. Fault analysis and debugging of microservice systems: Industrial survey, benchmark system, and empirical study[J]. IEEE Transactions on Software Engineering, 2021, 47(2): 243–260. doi: 10.1109/TSE.2018.2887384 [3] KHAN M G, TAHERI J, Al-DULAIMY A, et al. PerfSim: A performance simulator for cloud native microservice chains[J]. IEEE Transactions on Cloud Computing, To be published. [4] AROUK O and NIKAEIN N. Kube5G: A cloud-native 5G service platform[C]. 2020 IEEE Global Communications Conference, Taipei, China, 2020: 1–6. [5] GAO Xing, STEENKAMER B, GU Zhongshu, et al. A study on the security implications of information leakages in container clouds[J]. IEEE Transactions on Dependable and Secure Computing, 2021, 18(1): 174–191. doi: 10.1109/TDSC.2018.2879605 [6] NIFE F N and KOTULSKI Z. Application-aware firewall mechanism for software defined networks[J]. Journal of Network and Systems Management, 2020, 28(3): 605–626. doi: 10.1007/s10922-020-09518-z [7] BARDAS A G, SUNDARAMURTHY S C, OU Xinming, et al. MTD CBITS: Moving target defense for cloud-based IT systems[C]. The 22nd European Symposium on Research in Computer Security, Oslo, Norway, 2017: 167–186. [8] LU Kangjie, SONG Chengyu, LEE B, et al. ASLR-guard: Stopping address space leakage for code reuse attacks[C]. The 22nd ACM SIGSAC Conference on Computer and Communications Security, Denver, USA, 2015: 280–291. [9] LARSEN P, BRUNTHALER S, DAVI L, et al. Automated Software Diversity[M]. Morgan & Claypool, 2015: 1–8. [10] MEIER R, TSANKOV P, LENDERS V, et al. NetHide: Secure and practical network topology obfuscation[C]. The 27th USENIX Security Symposium, Baltimore, USA, 2018: 1–18. [11] JIN Hai, LI Zhi, ZOU Deqing, et al. DSEOM: A framework for dynamic security evaluation and optimization of MTD in container-based cloud[J]. IEEE Transactions on Dependable and Secure Computing, 2021, 18(3): 1125–1136. doi: 10.1109/TDSC.2019.2916666 [12] GLUCK A. Introducing domain-oriented microservice architecture[EB/OL].https://eng.uber.com/microservice-architecture, 2021. [13] NIST. National vulnerability database[EB/OL]. https://nvd.nist.gov/vuln, 2021. [14] PENG Wei, LI Feng, HUANG C T, et al. A moving-target defense strategy for cloud-based services with heterogeneous and dynamic attack surfaces[C]. 2014 IEEE International Conference on Communications, Sydney, Australia, 2014: 804–809. [15] 邱航, 汤红波, 游伟. 基于深度Q网络的在线服务功能链部署方法[J]. 电子与信息学报, 2021, 43(11): 3122–3130. doi: 10.11999/JEIT201009QIU Hang, TANG Hongbo, and YOU Wei. Online service function chain deployment method based on deep Q network[J]. Journal of Electronics &Information Technology, 2021, 43(11): 3122–3130. doi: 10.11999/JEIT201009 -
下载:
点击查看大图
图(4) / 表(2)
计量
- 文章访问数: 727
- HTML全文浏览量: 270
- PDF下载量: 175
- 被引次数: 0
