Video scene switching detection is important in video processing. An excellent algorithm for video scene switching detection is of great significance for subsequent video processing, including information annotation and feature extraction. In this study, on the basis of the traditional image processing methods and oriented FAST and rotated BRIEF (ORB), the similarity estimation of images is realized separately. Through the dynamic fusion factor, the value of similarity obtained by various basic algorithms in the complete algorithm is quantified, and the dynamic threshold is calculated according to the similarity of neighboring frames to improve the stability of the algorithm. In addition, by flow planning, the number of calls of the time-consuming part of the algorithm is reduced, and efficient video scene switching detection is finally realized. The experimental results show that the accuracy of video scene switching detection is improved to a certain extent.