In view of the handwritten equation with complex two-dimensional spatial structure in the mathematics class of primary and secondary schools, this study proposes a solution of off-line handwritten equation recognition based on multiple geometric features and Convolutional Neural Network (CNN). First, based on CNN classification algorithm, the single handwritten character is recognized after image preprocessing. Then, using geometric features, such as aspect ratio, center of mass coordinate, center of mass offset angle, center offset, horizontal overlap interval ratio, etc., to recognize common handwritten formulas such as decimal, fraction, index, and root formula with complex spatial structure, and using the divide-and-conquer algorithm to complete the recognition of composite formulas nested by the above formula combination. Finally, the off-line handwritten arithmetic recognition system is designed and implemented. The experimental results show that under certain illumination conditions, the recognition rate of handwritten equation of different resolutions and noisy images can reach 90.43%, which has certain application value.