After the occurrence of destructive earthquake, compared with various urban public facilities and residential buildings built in accordance with the relatively seismic fortification standards, the vast number of villages and towns without seismic fortification houses are more likely to collapse or even completely damage. In the past, earthquake disaster risk investigation and disaster assessment relied on the field survey of experts to determine the number and proportion of buildings of different structural types. In this study, brick (concrete) wood structure houses are identified by deep learning and photography technology. The Faster R-CNN model is trained for the data set of brick (mixed) wood houses in Shandong Province of Tan Lu fault zone, with an average accuracy of 91.868%. The results show that this method can effectively detect brick (concrete) wood houses, and can be applied to earthquake disaster pre-assessment, earthquake disaster risk investigation, earthquake key-risk area investigation, and other related work.