At the present stage, fires occur frequently, and fire detection and identification are required automatically. Although there are fire detection methods such as temperature and smoke sensors, the real-time detection is not guaranteed. To solve this problem, a method based on improved YOLOv3 fire detection and identification is proposed. Firstly, a multi-scenario large-scale fire target detection database was constructed to mark the categories and locations of the flame and smoke areas, and the problem of insufficient performance of YOLOv3 small target recognition was improved. Combined with the feature extraction ability of deep network, the fire detection and recognition were formalized into multi-class recognition and coordinate regression problems. The detection and recognition models of flame and smoke were obtained under different scenarios. Experiments show that the improved YOLOv3 algorithm proposed in this study can achieve ideal results for flame and smoke detection under different shooting angles and different illumination conditions, and also meets the real-time detection requirements in terms of detection speed.