Abstract: Extracting an effective visual feature to uncover semantic information is an important work for designing a robust automatic image annotation system. Since different kinds of heterogeneous features (such as color, texture and shape) show different intrinsic discriminative power and the same kind of features are usually correlated for image understanding, a Graph Regularized Non-negative Group Sparsity (GRNGS) model for image annotation is proposed, which can be effectively solved by a new method of non-negative matrix factorization. This model combines graph regularization withl2,1-norm regularization, and is able to select proper group features, which can describe both visual similarities and semantic correlations when performing the task of image annotation. Experimental results reported over the Corel5K and ESP Game databases show the robust capability and good performance of the proposed method.