Image feature extraction is always the core task of computer vision and image processing. With the rapid development of deep learning, the Convolutional Neural Network (CNN) has gradually replaced the traditional image feature operator and became the main algorithm for feature extraction. Combined with CNN and sum pooling, we propose a new image feature extraction algorithm based on depth prior aiming at the data association problem in the crowd sourcing labeling system for urban remote sensing data. The feature can effectively focus on the objects in the vicinity of outdoor images and verify their good characterization of outdoor images via image retrieval experiments.