Abstract: It is difficult for the traditional projection algorithms to extend to incremental learning since they use the whole training sets for solving out the projection matrix directly. To tackle this problem, a novel method, named Block optimization and Combination strategy, used for Locally Discriminant Projection (BCLDP) is proposed. This method takes into account both intra-class and interclass geometries; and has the orthogonality property. Furthermore, BCLDP is extended to nonlinear case using kernel function, which makes BCLDP better suits for diverse application. The experiments on ORL face database and speaker identification application demonstrate the effectiveness of the proposed algorithm.