Abstract: Scene image classification refers to the task of grouping different images into semantic categories. A new spatial semantic objects-based hybrid learning method is proposed to overcome the disadvantages existing in most of the relative methods. This method uses generative model to deal with the objects obtained by multi-scale segmentation instead of whole image, and calculates kinds of visual features to mine the category information of every objects. Then, an intermediate vector is generated using spatial-pyramid matching algorithm, to describe both the layer data and semantic information and narrow down the semantic gap. The method also combines a discriminative learning procedure to train a more confident classifier. Experimental results demonstrate that the proposed method can achieve high training efficiency and classification accuracy in interpreting manifold and complicated images.