Theoretical Analysis of Direct LDA in Small Sample Size Problem

Zhao Wu-feng; Shen Hai-bin; Yan Xiao-lang

doi:10.3724/SP.J.1146.2008.01629

Volume 31 Issue 11

Dec. 2010

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2009 > 31(11): 2632-2636

Zhao Wu-feng, Shen Hai-bin, Yan Xiao-lang. Theoretical Analysis of Direct LDA in Small Sample Size Problem[J]. Journal of Electronics & Information Technology, 2009, 31(11): 2632-2636. doi: 10.3724/SP.J.1146.2008.01629

Citation:

Zhao Wu-feng, Shen Hai-bin, Yan Xiao-lang. Theoretical Analysis of Direct LDA in Small Sample Size Problem[J]. Journal of Electronics & Information Technology, 2009, 31(11): 2632-2636. doi: 10.3724/SP.J.1146.2008.01629

Citation:

PDF( 219 KB)

Theoretical Analysis of Direct LDA in Small Sample Size Problem

doi: 10.3724/SP.J.1146.2008.01629

Zhao Wu-feng^{①②;沈海斌},
Shen Hai-bin,
Yan Xiao-lang

Received Date: 2008-12-05
Rev Recd Date: 2009-06-01
Publish Date: 2009-11-19

Abstract

Abstract

Direct LDA (DLDA) is an extension of Linear Discriminant Analysis (LDA) to deal with the small sample size problem, which is previously claimed to take advantage of all the information, both within and outside of the within-class scatter's null space. However, a lot of counter-examples show that this is not the case. In order to better understand the characteristics of DLDA, this paper presents its theoretical analysis and concludes that: DLDA based on the traditional Fisher criterion nearly does not make use of the information inside the null space, thus some discriminative information may be lost; while one based on other variants of Fisher criterion is equivalent to null-space LDA and orthogonal LDA under the orthogonal constraints among discriminant vectors and a mild condition which holds in many applications involving high-dimensional data. The comparative results on the face database, ORL and YALE, also consistent with the theory analysis.

FullText(HTML)

References(1)

References

Krzanowski W J, Jonathan P, and Mccarthy W V, et al..Discriminant analysis with singular covariance matrices:methods and applications to spectroscopic data. AppliedStatistics, 1995, 44(11): 101-115.[2]Belhumeur P N, Hespanha J P, and Kriegman D J.Eigenfaces vs. Fisherfaces: Recognition using class specificlinear projection. IEEE Transactions on Pattern Analysisand Machine Intelligence, 1997, 19(7): 711-720.[3]Yu Hua and Yang Jie. A direct LDA algorithm for highdimensionaldata-with application to face recognition[J].Pattern Recognition.2001, 34(10):2067-2070[4]Chen L F, Liao H Y M, and Ko M T, et al.. A newLDA-based face recognition system which can solve thesmall sample size problem[J].Pattern Recognition.2000,33(10):1713-1726[5]Huang R, Liu Q, and Lu H, et al.. Solving the small samplesize problem of LDA. Proceedings of InternationalConference on Pattern Recognition, USA, 2002, 3: 29-32.[6]Ye Jie-ping. Characterization of a family of algorithms forgeneralized discriminant analysis on undersampledproblems. Journal of Machine Learning Research, 2005,6(Apr.): 483-502.[7]Ye Jie-ping and Xiong Tao. Computational and theoreticalanalysis of null space and orthogonal linear discriminantanalysis. Journal of Machine Learning Research, 2006,7(Jul.): 1183-1204.[8]Zheng Yujie, Guo Zhibo, and Yang Jian, et al.. DLDA/QR:A robust direct LDA algorithm for face recognition and itstheoretical foundation[J].Lecture Notes in Computer Science.2007, 4426:379-387[9]Park C H and Lee M. On applying linear discriminantanalysis for multi-labeled problems[J].Pattern RecognitionLetter.2008, 29(7):878-887[10]厉小润, 赵光宙, 赵辽英. 改进的核直接Fisher描述分析与人脸识别. 浙江大学学报(工学版), 2008, 42(4): 583-589.Li Xiao-run, Zhao Guang-zhou, and Zhao Liao-ying.Improved kernel direct Fisher discriminant analysis andface recognition. Journal of Zhejiang University(Engineering Science), 2008, 42(4): 583-589.[11]Cevikalp H, Neamtu M, and Wilkes M, et al..Discriminative common vector method with kernels[J].IEEETransactions on Neural Networks.2006, 17(6):1550-1565[12]Park H and Park C H. A comparison of generalized lineardiscriminant analysis algorithms[J].Pattern Recognition.2008, 41(3):1083-1097[13]Gao Hui and Davis J W. Why direct LDA is not equivalentto LDA[J].Pattern Recognition.2006, 39(5):1002-1006

Relative Articles

Supplements(0)

Cited By

Proportional views