Abstract: Aiming at the problem that the traditional structure extraction method is difficult to maintain the structure while smoothing the texture, the iterative reweighted least squares optimization framework is used to construct an optimization model. In view of the difference in the gradient of texture and structure in the image, a relative extremum variation is proposed. Firstly, the extremum information of the gradient map is extracted through closed operation in morphology. And then, the index weight of windowed inherent variation is expanded, and then a new optimization model is constructed. Finally, according to the non-convexity of the model, a Toeplitz matrix is constructed for numerical solution. The experimental results show that the proposed algorithm can effectively remove the image texture, better preserve the edge structure and contrast, and also shows better advantages in image quality analysis.