在小样本条件下直接LDA的理论分析

赵武锋; 沈海斌; 严晓浪

doi:10.3724/SP.J.1146.2008.01629

在小样本条件下直接LDA的理论分析

doi: 10.3724/SP.J.1146.2008.01629

赵武锋^{①②;沈海斌},
沈海斌,
严晓浪

计量
- 文章访问数: 3771
- HTML全文浏览量: 160
- PDF下载量: 1219
- 被引次数: 0
出版历程
- 收稿日期: 2008-12-05
- 修回日期: 2009-06-01
- 刊出日期: 2009-11-19

Theoretical Analysis of Direct LDA in Small Sample Size Problem

Zhao Wu-feng^{①②;沈海斌},
Shen Hai-bin,
Yan Xiao-lang

摘要

摘要: 直接线性鉴别分析(DLDA)是一种以克服小样本问题而提出的LDA扩展方法，被声明利用了包含类内散布矩阵零空间外的所有信息。然而，很多反例表明事实并非如此。为了更深入地了解DLDA的特性，该文从理论上对其进行了分析，得出结论：基于传统Fisher准则的DLDA几乎没利用零空间，将丢失一些有用的鉴别信息；而基于广义Fisher准则的DLDA，若满足一定条件(在高维小样本数据应用中一般都满足)且最优鉴别矢量正交约束，则其等价于零空间LDA和正交LDA。在人脸数据库ORL和YALE上的比较实验结果亦与理论分析一致。
- 模式识别; Fisher准则; 降维; 线性鉴别分析; 小样本
Abstract: Direct LDA (DLDA) is an extension of Linear Discriminant Analysis (LDA) to deal with the small sample size problem, which is previously claimed to take advantage of all the information, both within and outside of the within-class scatter's null space. However, a lot of counter-examples show that this is not the case. In order to better understand the characteristics of DLDA, this paper presents its theoretical analysis and concludes that: DLDA based on the traditional Fisher criterion nearly does not make use of the information inside the null space, thus some discriminative information may be lost; while one based on other variants of Fisher criterion is equivalent to null-space LDA and orthogonal LDA under the orthogonal constraints among discriminant vectors and a mild condition which holds in many applications involving high-dimensional data. The comparative results on the face database, ORL and YALE, also consistent with the theory analysis.

HTML全文

参考文献(1)

Krzanowski W J, Jonathan P, and Mccarthy W V, et al..Discriminant analysis with singular covariance matrices:methods and applications to spectroscopic data. AppliedStatistics, 1995, 44(11): 101-115.[2]Belhumeur P N, Hespanha J P, and Kriegman D J.Eigenfaces vs. Fisherfaces: Recognition using class specificlinear projection. IEEE Transactions on Pattern Analysisand Machine Intelligence, 1997, 19(7): 711-720.[3]Yu Hua and Yang Jie. A direct LDA algorithm for highdimensionaldata-with application to face recognition[J].Pattern Recognition.2001, 34(10):2067-2070[4]Chen L F, Liao H Y M, and Ko M T, et al.. A newLDA-based face recognition system which can solve thesmall sample size problem[J].Pattern Recognition.2000,33(10):1713-1726[5]Huang R, Liu Q, and Lu H, et al.. Solving the small samplesize problem of LDA. Proceedings of InternationalConference on Pattern Recognition, USA, 2002, 3: 29-32.[6]Ye Jie-ping. Characterization of a family of algorithms forgeneralized discriminant analysis on undersampledproblems. Journal of Machine Learning Research, 2005,6(Apr.): 483-502.[7]Ye Jie-ping and Xiong Tao. Computational and theoreticalanalysis of null space and orthogonal linear discriminantanalysis. Journal of Machine Learning Research, 2006,7(Jul.): 1183-1204.[8]Zheng Yujie, Guo Zhibo, and Yang Jian, et al.. DLDA/QR:A robust direct LDA algorithm for face recognition and itstheoretical foundation[J].Lecture Notes in Computer Science.2007, 4426:379-387[9]Park C H and Lee M. On applying linear discriminantanalysis for multi-labeled problems[J].Pattern RecognitionLetter.2008, 29(7):878-887[10]厉小润, 赵光宙, 赵辽英. 改进的核直接Fisher描述分析与人脸识别. 浙江大学学报(工学版), 2008, 42(4): 583-589.Li Xiao-run, Zhao Guang-zhou, and Zhao Liao-ying.Improved kernel direct Fisher discriminant analysis andface recognition. Journal of Zhejiang University(Engineering Science), 2008, 42(4): 583-589.[11]Cevikalp H, Neamtu M, and Wilkes M, et al..Discriminative common vector method with kernels[J].IEEETransactions on Neural Networks.2006, 17(6):1550-1565[12]Park H and Park C H. A comparison of generalized lineardiscriminant analysis algorithms[J].Pattern Recognition.2008, 41(3):1083-1097[13]Gao Hui and Davis J W. Why direct LDA is not equivalentto LDA[J].Pattern Recognition.2006, 39(5):1002-1006

施引文献

资源附件(0)

访问统计