Abstract: In this paper, the deformable convolution module is introduced to enhance the modeling ability of the network to digital geometric transformation, and an improved handwritten digital recognition framework based on deformable CNN is proposed. In addition to improving the recognition accuracy, the framework can effectively reduce the training parameters and improve the recognition speed. This paper demonstrate state-of-the-art performance competing methods on the handwritten dataset and the transformed dataset.