Abstract: The Bag of Words (BoW) model is applied to object classification in this paper. An optimized method based on the combination of Region Of Interest (ROI) extraction and pyramid matching scheme is proposed to optimize and improve the traditional model in order to overcome the disadvantages. First, the ROI is extracted from training images and then the codebook is generated using the features which are extracted from the ROI instead of the entire images using dense Scale-Invariant Feature Transform (SIFT) descriptor. Therefore, the codebook can describe the features of the images more accurately and also can resist the impact of the various position information as well as background. Then the images will be represented as the histogram of codebook using pyramid matching scheme as the input of Support Vector Machine (SVM) classifier. The experiments are carried out based both Caltech 101 and Caltech 256 database. The results show that the proposed method performs better than the traditional method and the state of the art. What is more, the classification accuracy is good even though under lack of training images.