Optimal Mean Linear Classifier via Weighted Nuclear Norm and <i>L</i><sub>2,1</sub> Norm

ZENG Deyu; LIANG Zexiao; WU Zongze

doi:10.11999/JEIT211434

Volume 44 Issue 5

May 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 > 44(5): 1602-1609

ZENG Deyu, LIANG Zexiao, WU Zongze. Optimal Mean Linear Classifier via Weighted Nuclear Norm and L2,1 Norm[J]. Journal of Electronics & Information Technology, 2022, 44(5): 1602-1609. doi: 10.11999/JEIT211434

Citation:

ZENG Deyu, LIANG Zexiao, WU Zongze. Optimal Mean Linear Classifier via Weighted Nuclear Norm and L_2,1 Norm[J]. Journal of Electronics & Information Technology, 2022, 44(5): 1602-1609. doi: 10.11999/JEIT211434

Citation:

PDF( 1873 KB)

Optimal Mean Linear Classifier via Weighted Nuclear Norm and L_2,1 Norm

doi: 10.11999/JEIT211434

1.
School of Automation, Guangdong University of Technology, Guangzhou 510006, China
2.
Guangdong-HongKong-Macao Joint Laboratory for Smart Discrete Manufacturing, Guangdong University of Technology, Guangzhou 510006, China

Funds: Guangdong Province Key Field R&D Program (2021B0101200005), The National Natural Science Foundation of China (62073088, U1911401), Guangdong Province Basic and Applied Basic Research Fund (2019A1515011606)

Received Date: 2021-12-06
Rev Recd Date: 2022-04-14

Available Online: 2022-04-21

Publish Date: 2022-05-25

Abstract

Abstract

Defect detection is an important part of intelligent manufacturing system. When traditional machine learning algorithms are used for defect classification, data noise interference is usually encountered, which reduces the algorithm’s prediction accuracy for defect classification. Although powerful algorithms such as Robust Linear Discriminant Analysis (RLDA) have been proposed in recent years to solve classification problems with data disturbed by sparse noise, there are still some drawbacks that limit its application performance. In this paper, a new Optimal Mean-Robust Linear Classification Analyis (OMRLSA) based on linear discriminant analysis is proposed. Different from the previous classification methods dealing with noisy data, ignoring the influence of the Laplace distribution characteristic of sparse noise on the data mean, the optimal mean robust linear classification model proposed in this paper will automatically update the optimal mean of the data. This ensures that the statistical characteristics of the data will not be disturbed by noise. Furthermore, a weighted kernel norm minimization method with joint L_2,1 norm minimization and rank compression on regularization and error measurement is introduced for the first time in a robust classification model in the subsequent loss function. Thereby the robustness of the algorithm is improved. Experimental results on standard dataset with different ratio corruption illustrate the superiority of the proposed method.
- Defect detection,
- Defect classification,
- Robust linear classification,
- Low rank recovery

FullText(HTML)

References(39)

References

[1]	孙洪宇, 彭丽莎, 屈凯峰, 等. 机器学习在复合绝缘子缺陷超声检测中的应用与展望[J]. 无损检测, 2021, 43(5): 58–63. doi: 10.11973/wsjc202105013 SUN Hongyu, PENG Lisha, QU Kaifeng, et al. Application and prospect of machine learning in ultrasonic testing of composite insulator defects[J]. Nondestructive Testing, 2021, 43(5): 58–63. doi: 10.11973/wsjc202105013
[2]	唐伦, 廖皓, 曹睿, 等. 基于深度动态贝叶斯网络的服务功能链故障诊断算法[J]. 电子与信息学报, 2021, 43(12): 3588–3596. doi: 10.11999/JEIT200029 TANG Lun, LIAO Hao, CAO Rui, et al. Fault diagnosis algorithm of service function chain based on deep dynamic Bayesian network[J]. Journal of Electronics &Information Technology, 2021, 43(12): 3588–3596. doi: 10.11999/JEIT200029
[3]	王剑, 匡洪宇, 李瑞林, 等. 基于CNN-GAP可解释性模型的软件源码漏洞检测方法[J]. 电子与信息学报, 待发表. doi: 10.11999/JEIT210412. WANG Jian, KUANG Hongyu, LI Ruilin, et al. . Software source code vulnerability detection based on CNN-GAP interpretability model[J]. Journal of Electronics & Information Technology, To be published. doi: 10.11999/JEIT210412.
[4]	刘华祠. 基于传统机器学习与深度学习的图像分类算法对比分析[J]. 电脑与信息技术, 2019, 27(5): 12–15. doi: 10.19414/j.cnki.1005-1228.2019.05.004 LIU Huaci. Comparative analysis of image classification algorithms based on traditional machine learning and deep learning[J]. Computer and Information Technology, 2019, 27(5): 12–15. doi: 10.19414/j.cnki.1005-1228.2019.05.004
[5]	刘浩, 陈再良, 张良. 基于机器学习的刀具表面缺陷检测及分类方法[J]. 计算机测量与控制, 2021, 29(6): 64–68. doi: 10.16526/j.cnki.11-4762/tp.2021.06.014 LIU Hao, CHEN Zailiang, and ZHANG Liang. Tool surface defect detection and classification based on machine learning[J]. Computer Measurement &Control, 2021, 29(6): 64–68. doi: 10.16526/j.cnki.11-4762/tp.2021.06.014
[6]	李宇庭, 甘芳吉, 万正军, 等. 基于SVM及电流牵扯效应的金属缺陷分类识别方法[J]. 四川大学学报(工程科学版), 2015, 47(6): 172–178. doi: 10.15961/j.jsuese.2015.06.024 LI Yuting, GAN Fangji, WAN Zhengjun, et al. Method for defect classification based on SVM and current drag effect[J]. Journal of Sichuan University (Engineering Science Edition), 2015, 47(6): 172–178. doi: 10.15961/j.jsuese.2015.06.024
[7]	舒文娉, 刘全香. 基于支持向量机的印品缺陷分类方法[J]. 包装工程, 2014, 35(23): 138–142. doi: 10.19554/j.cnki.1001-3563.2014.23.028 SHU Wenping and LIU Quanxiang. Classification method of printing defects based on support vector machine[J]. Packaging Engineering, 2014, 35(23): 138–142. doi: 10.19554/j.cnki.1001-3563.2014.23.028
[8]	HAGI H, IWAHORI Y, FUKUI S, et al. Defect classification of electronic circuit board using SVM based on random sampling[J]. Procedia Computer Science, 2014, 35: 1210–1218. doi: 10.1016/j.procs.2014.08.218
[9]	WRIGHT J, YANG A Y, GANESH A, et al. Robust face recognition via sparse representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 31(2): 210–227. doi: 10.1109/TPAMI.2008.79
[10]	CHEN Yudong, CARAMANIS C, and MANNOR S. Robust sparse regression under adversarial corruption[C]. The 30th International Conference on International Conference on Machine Learning, Atlanta, USA, 2013: III-774–III-782.
[11]	NIE Feiping, HUANG Heng, CAI Xiao, et al. Efficient and robust feature selection via joint ℓ2, 1-norms minimization[C]. The 23rd International Conference on Neural Information Processing Systems, Red Hook, USA, 2010: 1813–1821.
[12]	SHI Xiaoshuang, YANG Yujiu, GUO Zhenhua, et al. Face recognition by sparse discriminant analysis via joint L2, 1-norm minimization[J]. Pattern Recognition, 2014, 47(7): 2447–2453. doi: 10.1016/j.patcog.2014.01.007
[13]	VAPNIK V. Statistical Learning Theory[M]. New York, USA: Wiley, 1998.
[14]	MARONNA R A, MARTIN R D, and YOHAI V J. Robust Statistics: Theory and Methods[M]. Chichester: John Wiley & Sons, Ltd, 2006.
[15]	李松涛, 李维刚, 甘平, 等. 基于Sinkhorn距离特征缩放的多约束非负矩阵分解算法[J/OL]. 电子与信息学报, 待发表. doi: 10.11999/JEIT210946 LI Songtao, LI Weigang, GAN Ping, et al. Multi-constrained non-negative matrix factorization algorithm based on sinkhorn distance feature scaling[J/OL]. Journal of Electronics & Information Technology, To be published. doi: 10.11999/JEIT210946.
[16]	李相平, 王明泽, 但波, 等. 基于鲁棒主成分分析的多域联合杂波抑制算法[J]. 电子与信息学报, 2022, 44(4): 1303–1310. doi: 10.11999/JEIT210676 LI Xiangping, WANG Mingze, DAN Bo, et al. The multi-domain union clutter suppression algorithm based on robust principal component analysis[J]. Journal of Electronics &Information Technology, 2022, 44(4): 1303–1310. doi: 10.11999/JEIT210676
[17]	唐红梅, 白梦月, 韩力英, 等. 基于低秩背景约束与多线索传播的图像显著性检测[J]. 电子与信息学报, 2021, 43(5): 1432–1440. doi: 10.11999/JEIT200193 TANG Hongmei, BAI Mengyue, HAN Liying, et al. Image saliency detection based on background constraint of low rank and multi-cue propagation[J]. Journal of Electronics &Information Technology, 2021, 43(5): 1432–1440. doi: 10.11999/JEIT200193
[18]	CANDÈS E J, LI Xiaodong, MA Yi, et al. Robust principal component analysis?[J] Journal of the ACM, 2009, 58(3): 11.
[19]	DING C, ZHOU Ding, HE Xiaofeng, et al. R1-PCA: Rotational invariant L1-norm principal component analysis for robust subspace factorization[C].The 23rd International Conference on Machine Learning, New York, USA, 2006: 281–288.
[20]	YI Shuangyan, HE Zhenyu, and YANG Weiguo. Robust principal component analysis via joint ℓ2, 1-norms minimization[C]. 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), Shenzhen, China, 2017: 13–18.
[21]	NIE Feiping, HUANG Heng, and DING Chris HQ. Low-rank matrix recovery via efficient schatten p-norm minimization[C]. The Twenty-Sixth AAAI Conference on Artificial Intelligence, Toronto, Canada, 2012: 655–661.
[22]	NIE Feiping, WANG Hua, CAI Xiao, et al. Robust matrix completion via joint schatten p-norm and ℓ_p-norm minimization[C]. 2012 IEEE 12th International Conference on Data Mining, Brussels, Belgium, 2012: 566–574.
[23]	HE Ran, HU Baogang, ZHENG Weishi, et al. Robust principal component analysis based on maximum correntropy criterion[J]. IEEE Transactions on Image Processing, 2011, 20(6): 1485–1494. doi: 10.1109/TIP.2010.2103949
[24]	NIE Feiping, YUAN Jianjun, and HUANG Heng. Optimal mean robust principal component analysis[C]. The 31st International Conference on International Conference on Machine Learning, Beijing, China, 2014: II-1062–II-1070.
[25]	SHI Xiaoshuang, NIE Feiping, LAI Zhihui, et al. Robust principal component analysis via optimal mean by joint ℓ2, 1 and schatten p -norms minimization[J]. Neurocomputing, 2018, 283: 205–213. doi: 10.1016/j.neucom.2017.12.034
[26]	LIANG Yiniang, REN Zhigang, WU Zongze, et al. Scalable spectral ensemble clustering via building representative co-association matrix[J]. Neurocomputing, 2020, 390: 158–167. doi: 10.1016/j.neucom.2020.01.055
[27]	LIANG Zexiao, GUO Shaozhi, LIU Dakang, et al. Spectral clustering based on high-frequency texture components for face datasets[J]. IET Image Processing, 2021, 15(10): 2240–2246. doi: 10.1049/IPR2.12191
[28]	WU Zongze, LIU Sihui, DING C, et al. Learning graph similarity with large spectral gap[J]. IEEE Transactions on Systems, Man, and Cybernetics:Systems, 2021, 51(3): 1590–1600. doi: 10.1109/TSMC.2019.2899398
[29]	WU Zongze, SU Chuncheng, YIN Ming, et al. Subspace clustering via stacked independent subspace analysis networks with sparse prior information[J]. Pattern Recognition Letters, 2021, 146: 165–171. doi: 10.1016/j.patrec.2021.03.026
[30]	GOLDBERG A B, ZHU Xiaojin, RECHT Ben, et al. Transduction with matrix completion: Three birds with one stone[C]. The 23rd International Conference on Neural Information Processing Systems, Red Hook, USA, 2010: 757–765.
[31]	CABRAL R, DE LA TORRE F, COSTEIRA J P, et al. Matrix completion for weakly supervised multi-label image classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(1): 121–135. doi: 10.1109/TPAMI.2014.2343234
[32]	HUANG Dong, CABRAL R, and DE LA TORRE F. Robust regression[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 363–375. doi: 10.1109/TPAMI.2015.2448091
[33]	ZENG Deyu, WU Zongze, DING C, et al. Labeled-robust regression: Simultaneous data recovery and classification[J]. IEEE Transactions on Cybernetics, To be published. doi: 10.1109/TCYB.2020.3026101
[34]	WRIGHT J, PENG Yigang, MA Yi, et al. Robust principal component analysis: Exact recovery of corrupted low-rank matrices by convex optimization[C]. The 22nd International Conference on Neural Information Processing Systems, Red Hook, USA, 2009: 2080–2088.
[35]	LIN Zhouchen, CHEN Minming, WU Leqin, et al. The augmented Lagrange multiplier method for exact recovery of corrupted low rank matrices[R]. UILU-ENG-09–2215, 2009.
[36]	YIN Ming, ZENG Deyu, GAO Junbin, et al. Robust multinomial logistic regression based on RPCA[J]. IEEE Journal of Selected Topics in Signal Processing, 2018, 12(6): 1144–1154. doi: 10.1109/JSTSP.2018.2872460
[37]	GU Shuhang, XIE Qi, MENG Deyu, et al. Weighted nuclear norm minimization and its applications to low level vision[J]. International Journal of Computer Vision, 2017, 121(2): 183–208. doi: 10.1007/s11263-016-0930-5
[38]	CAI Jianfeng, CANDÈS E J, and SHEN Zuowei. A singular value thresholding algorithm for matrix completion[J]. SIAM Journal on Optimization, 2008, 20(4): 1956–1982. doi: 10.1137/080738970
[39]	ZHANG Zheng, XU Yong, SHAO Ling, et al. Discriminative block-diagonal representation learning for image recognition[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 29(7): 3111–3125. doi: 10.1109/TNNLS.2017.2712801