In recent years, the Kernel Correlation Filtering algorithm (KCF algorithm) proposed by Henriques et al. shows superior performance in terms of algorithm scale, computational complexity, and algorithm performance. Based on KCF algorithm, a target tracking system based on DSP is proposed and designed in this study. In terms of hardware, this study designs and implements a complete and independent hardware platform. In terms of software, this study proposes a series of algorithm optimization methods for DSP to optimize KCF algorithm, in order to meet the requirements of important engineering indicators. The results show that the system performs well in the actual engineering environment, the highest tracking angular velocity can be 20 degrees/s, and the frame rate can be 25 fps on average, and it has high accuracy. The system provides reference for embedded applications of various algorithms in the field of computer vision.