Abstract: Image segmentation is executed firstly by using Ward clustering algorithm, the result of which is served as initial value of the Markov random field (MRF) model by applying the spatial neighborhood information to complete image classification.The priori probability of the classification of the image pixel is built on Ising model, the finite Gaussian mixture (FGM) model is used to describe the conditional probability distribution of the image intensity.The expectation-maximization (EM) algorithm is applied in estimating the parameters of FGM model.The local optimization method of the iterative conditional modes (ICM) and the maximum a posteriori (MAP) method are used to estimate the image class label.Experiments demonstrate that the proposed algorithm is able to successfully segment various objects by compare with other algorithms.