Abstract: A new algorithm to detect the moving objects of video sequences based on time and spatial information is proposed in this paper.Firstly we operate the difference calculation to adjacent frames based on three-regulation, do binarization processing, so we can gain time information because of movement of the moving objects.Secondly sign adjacent pixels of the bilingual picture to remove the polluted pixels, then we can get models of the moving objects with the help of spatial information resulting from edge detection.At last the integral information will be obtained after connecting the outmost pixels of the models.The algorithm will detect the moving objects precisely according to using of the time information in combination with the spatial information of the moving objects.