With the popularity of video surveillance, action analysis technology based on artificial intelligence is playing an increasingly important role in the field of intelligent surveillance. Most of the existing algorithms depend on an optical flow network or a 3D network to obtain the time information of actions. However, the optical flow network and the general 3D network require a large amount of computation, and the computational efficiency is low when classification and localization are carried out simultaneously. To solve this problem, this study builds a dualflow framework capable of spatial localization and classification and follows the idea of SVD to decompose the 3D convolution kernel in the branch of the 3D network, thus reducing the 3D network parameters. In addition, the dynamic programming algorithm is employed to efficiently search the optimal action tubes, and the mixup algorithm is used to expand the data sets during training, thereby enhancing the training results. Finally, experimental verification is carried out on UCF101-24 and J-HMDB-21, two widely used data sets for action localization. Compared with the baseline algorithm, the Frame-mAP of the two data sets is improved by 7.1% and 4.8%, and the Video-mAP of J-HMDB-21 under different IoUs is enhanced by 5.2% and 4.8%. Experimental results show that the proposed algorithm can substantially improve the ability of action localization, with better results compared with other algorithms.