Abstract: Thanks to the rapid development and application of remote sensing technology and related monitoring technology, crop yield prediction has been more favored in this field in recent years since the band information was mined from remote sensing images.However, due to the limitation of space size and time difference, the multi band information affecting crop growth will be processed by various dimension reduction technologies, so the time-space and band characteristics of data are not fully utilized.Therefore, a deep learning architecture for crop yield prediction is proposed to solve these problems. This model combines 3D Convolution Network and Temporal Convolution Network to better capture the spatio-temporal information and band information of remote sensing images.In addition, a variable is introduced into the new loss function to eliminate the influence of unbalanced distribution of crop yield labels. Finally, the new model was validated by the prediction of maize yield data in China. The results were compared with the main depth learning methods. The experimental results show that the proposed method can provide better prediction performance than other competing methods.