Adversarial Training Defense Based on Second-order Adversarial Examples

Yaguan QIAN; Ximin ZHANG; Bin WANG; Zhaoquan GU; Wei LI; Bensheng YUN

doi:10.11999/JEIT200723

Volume 43 Issue 11

Nov. 2021

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2021 > 43(11): 3367-3373

Yaguan QIAN, Ximin ZHANG, Bin WANG, Zhaoquan GU, Wei LI, Bensheng YUN. Adversarial Training Defense Based on Second-order Adversarial Examples[J]. Journal of Electronics & Information Technology, 2021, 43(11): 3367-3373. doi: 10.11999/JEIT200723

Citation:

Yaguan QIAN, Ximin ZHANG, Bin WANG, Zhaoquan GU, Wei LI, Bensheng YUN. Adversarial Training Defense Based on Second-order Adversarial Examples[J]. Journal of Electronics & Information Technology, 2021, 43(11): 3367-3373. doi: 10.11999/JEIT200723

Citation:

PDF( 1061 KB)

Adversarial Training Defense Based on Second-order Adversarial Examples

doi: 10.11999/JEIT200723

1.
School of Science/School of Big-data Science, Zhejiang University of Science and Technology, Hangzhou 310023, China
2.
Network and Information Security Laboratory of Hangzhou Hikvision Digital Technology Co., Ltd. Hangzhou 310052, China
3.
Cyberspace Institute of Advanced Technology (CIAT), Guangzhou University, Guangzhou 510006, China

Funds: The National Research and Development Program of China (2018YFB2100400), The National Natural Science Foundation of China (61902082)

Received Date: 2020-08-06
Rev Recd Date: 2021-08-20

Available Online: 2021-09-16

Publish Date: 2021-11-23

Abstract

Abstract

Although Deep Neural Networks (DNN) achieves high accuracy in image recognition, it is significantly vulnerable to adversarial examples. Adversarial training is one of the effective methods to resist adversarial examples empirically. Generating more powerful adversarial examples can solve the inner maximization problem of adversarial training better, which is the key to improve the effectiveness of adversarial training. In this paper, to solve the inner maximization problem, an adversarial training based on second-order adversarial examples is proposed to generate more powerful adversarial examples through quadratic polynomial approximation in a tiny input neighborhood. Through theoretical analysis, second-order adversarial examples are shown to outperform first-order adversarial examples. Experiments on MNIST and CIFAR10 data sets show that second-order adversarial examples have high attack success rate and high concealment. Compared with PGD adversarial training, adversarial training based on second-order adversarial examples is robust to all the existing typical attacks.
- Adversarial examples,
- Adversarial training,
- The second-order Taylor expansion

FullText(HTML)

References(19)

References

[1]	CHICCO D, SADOWSKI P, and BALDI P. Deep autoencoder neural networks for gene ontology annotation predictions[C]. Proceedings of the 5th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics, Newport Beach, America, 2014: 533–554.
[2]	SPENCER M, EICKHOLT J, and CHENG Jianlin. A deep learning network approach to ab initio protein secondary structure prediction[J]. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2015, 12(1): 103–112. doi: 10.1109/TCBB.2014.2343960
[3]	MIKOLOV T, DEORAS A, POVEY D, et al. Strategies for training large scale neural network language models[C]. 2011 IEEE Workshop on Automatic Speech Recognition & Understanding, Waikoloa, America, 2011: 196–201.
[4]	HINTON G, DENG Li, YU Dong, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups[J]. IEEE Signal Processing Magazine, 2012, 29(6): 82–97. doi: 10.1109/MSP.2012.2205597
[5]	LECUN Y, KAVUKCUOGLU K, FARABET C, et al. Convolutional networks and applications in vision[C]. Proceedings of 2010 IEEE International Symposium on Circuits and Systems, Paris, France, 2010: 253–256.
[6]	KRIZHEVSKY A, SUTSKEVER I, and HINTON G E. ImageNet classification with deep convolutional neural networks[C]. Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, Lake Tahoe Nevada, America, 2012: 1097–1105.
[7]	SZEGEDY C, ZAREMBA W, SUTSKEVER I, et al. Intriguing properties of neural networks[C]. 2nd International Conference on Learning Representations, ICLR 2014, Banff, Canada, 2014.
[8]	STALLKAMP J, SCHLIPSING M, SALMEN J, et al. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition[J]. Neural Networks, 2012, 32: 323–332. doi: 10.1016/j.neunet.2012.02.016
[9]	CARLINI N and WAGNER D. Towards evaluating the robustness of neural networks[C]. 2017 IEEE Symposium on Security and Privacy (SP), San Jose, America, 2017: 39–57.
[10]	MOOSAVI-DEZFOOLI S M, FAWZI A, and FROSSARD P. DeepFool: A simple and accurate method to fool deep neural networks[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, America, 2016: 2574–2582. doi: 10.1109/CVPR.2016.282.
[11]	XIE Cihang, ZHANG Zhishuai, ZHOU Yuyin, et al. Improving transferability of adversarial examples with input diversity[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, America, 2019: 2725–2734.
[12]	LEE J G, JUN S, CHO Y W, et al. Deep learning in medical imaging: General overview[J]. Korean Journal of Radiology, 2017, 18(4): 570–584. doi: 10.3348/kjr.2017.18.4.570
[13]	MADRY A, MAKELOV A, SCHMIDT L, et al. Towards deep learning models resistant to adversarial attacks[C]. ICLR 2018 Conference Blind Submission, Vancouver, Canada, 2018.
[14]	GOODFELLOW I J, SHLENS J, and SZEGEDY C. Explaining and harnessing adversarial examples[C]. 3rd International Conference on Learning Representations, San Diego, America, 2015.
[15]	ARAUJO A, MEUNIER L, PINOT R, et al. Robust neural networks using randomized adversarial training[EB/OL]. https://arxiv.org/pdf/1903.10219.pdf, 2020.
[16]	LAMB A, BINAS J, GOYAL A, et al. Fortified networks: Improving the robustness of deep networks by modeling the manifold of hidden representations[C]. ICLR 2018 Conference Blind Submission, Vancouver, Canada, 2018.
[17]	XU Weilin, EVANS D, and QI Yanjun. Feature squeezing: Detecting adversarial examples in deep neural networks[C]. Network and Distributed Systems Security Symposium (NDSS), San Diego, America, 2018. doi: 10.14722/ndss.2018.23198.
[18]	BELINKOV Y and BISK Y. Synthetic and natural noise both break neural machine translation[C]. ICLR 2018 Conference Blind Submission, Vancouver, Canada, 2018.
[19]	YANG Yuzhe, ZHANG Guo, KATABI D, et al. ME-net: Towards effective adversarial robustness with matrix estimation[C]. Proceedings of the 36th International Conference on Machine Learning, Long Beach, America, 2019.