Abstract: It is always important issue to extract features that are most effective for preserving the distribution architecture in pattern recognition community. Kernel based methods are assumed to extract nonlinear features. However, it is very sensitive to the selection of its mapping function and parameters. This paper proposes a feature extraction algorithm based on multi-layer auto-encoder, which consists of two phases of unsupervised pretraining and supervised fine-tuning based on marginal Fisher rule. Generative pretraining and regularization methods within fine-tuning phase are adopted to avoid overfitting of models training. The validity of algorithm is proved within the result of classification experiments in several datasets.