As small targets in the early stage of a fire are difficult to detect during flames and smoke detection of fires, this study proposes an improved YOLOX-nano (ASe-YOLOX-nano) object detection algorithm based on natural exponential loss (eCIoU). Firstly, a new object detection function, the eIoU loss function, is proposed to replace the traditional IoU loss, which solves the problems of no intersection between the prediction box and the real frame in small target detection and the inability to react to the influence of width and height. Secondly, the attention module is introduced in the network model to vaguely locate the target position in the early stage of the network and improve the accuracy of the detection of targets, especially small targets, in the later stage of the network. In addition, the soft pooled spatial pyramid pooling structure is employed to extract spatial feature information of different sizes, which can improve the robustness of the model for spatial layout and object degeneration. In this way, sufficient features can be extracted when the target is small. Moreover, the Mosaic enhancement technology is used to preprocess the dataset to improve the generalization ability of the model for further improvement in network performance. The comparative verification of the target data set shows that the mAP index reaches 70.07%, which is 3.46% higher than that of the original model, and the model enjoys accuracy of flame and smoke detection of 84.66% and 74.56%, respectively, and a stable FPS of 73, which has better fire detection ability than the traditional YOLOX-nano algorithm.