Abstract: A new kernel logistic regression model based on two phase sparsity-promoting prior is proposed to render a sparse multi-classifier and enhance the run-time efficiency. For accelerating the building of the model, the bottom-up training algorithm is adopted which controls the capacity of the learned classifier by minimizing the number of basis functions used, resulting in better generalization and faster computation. Experimental results on standard benchmark data sets attest to the accuracy, sparsity, and efficiency of the proposed methods.