In view of the discontinuity and uncertainty of moving objects in the non overlapping view of multiple cameras, a handover algorithm of moving pedestrian target based on deep learning is proposed. Firstly, a face feature extraction model is constructed based on deep convolution neural network, and the face feature extraction model is trained to obtain accurate face features. Then compare two common similarity measurement methods, choose a more suitable similarity measurement method to complete the optimal face matching process, and then improve the accuracy of face matching. Finally, the most matching face can be found by feature matching under different cameras to realize the handover of moving objects. Experiments show that the depth neural network can accurately extract the facial features of moving objects, achieve the accurate matching process of human faces, and effectively complete the task of moving object tracking under multi cameras.