Face recognition in surveillance videos is an essential technology in public security and has gotten more and more attention. But it is a little hard for the face recognition systems to be integrated into real application due to the low recognition rate caused partly by low face image quality. This study proposes a method of face image quality assessment using CNN. The proposed net, modified from the Alexnet, connects intermediate convolution layers to fully connect layer, to get multiple image features. Then, face image quality scores can be gotten from proposed net which is trained by end to end. In addition, a face image quality metric is used to relate the quality with the face recognition algorithm. Experiments on Color FERET datasets show that the proposed algorithm is able to elevate the face image quality exactly. Further experiments on a video surveillance dataset (collected by ourselves) show that the proposed method can select high quality face image for face recognition, leading to significant improvements in recognition accuracy.