Abstract: A moving object detection algorithm based on the theory of dictionary learning is proposed.Firstly,the algorithm gets an initial background image from the training samples by multi-frame averaging algorithm,and then the initial background sparse representation model is built upon it by BP algorithm.Secondly,combining with the current adjacent five frames,the dictionary is updated adaptively by K-SVD method in order to make the background model approximate adjacent frames background's observation values optimally.Finally,the foreground moving object is obtained by subtracting the background model from the current image.Simulation and comparison experimental results demonstrate that the algorithm can not only reduce data redundancy effectively and decrease the running time via sparse representation,but also can obtain a more robust background dictionary and avoid the interference of the pseudo-foreground by making full advantage of correlation of adjacent frames,and in the end increased the precision rate of moving object detection.