Abstract: The feedback of the content-based image retrieval system with the SVM algorithm, In order to improve the accuracy of image retrieval and reduce the number of user feedback, we proposed a novel algorithm called KRSVM, extracting features of the image about color shape and shape by using color moments, color histograms and Hu moments, and they are clustered with the multiple clusters of the K-means algorithm, at the same time, the SVM algorithm trains the image samples using a number of SVM kernel functions and Relief algorithm, they are for the weight distribution of the kernel functions and the image features. Compared with the traditional SVM algorithm and K-SVM algorithm, this algorithm is more better in terms of accuracy and requirements to some extent.