Abstract: Kernel-based Support Vector Machine (SVM) is widely used in many fields ( e.g. image classification) for its good generalization, in which the key factor is to design effective kernel functions. As there is not much a priori knowledge introduced into traditional kernel functions, the data-driven kernel building method is proposed to construct a new histogram kernel function which is combined with Bag OF Word (BOW) model and based on TF-IDF Weighted Quadratic Chi-squared (WQC) distance. In the process of calculating distances between histograms, the distinct discriminative power of each histogram bin is fully taken into consideration to boost classification performance of kernel functions. Experiments on several classic image data sets (Caltech101/256, etc.) show the better classification performance of the proposed method.