Abstract:Intra-modes decision in HEVC has higher complexity. Aiming at this problem, after brief analysis of the intra prediction algorithm, an intra prediction method selection algorithm based on the improved Canny operator is proposed to reduce the HEVC intra-frame prediction complexity. Firstly, The improved Canny operator is used to obtain the direction of the Prediction Unites(PU) in advance, in order to select the corresponding angle prediction model to comprise the candidate list，and the spatial correlation of images is adopted to add the optimal mode of the neighboring PU to the candidate list. At Last, this algorithm takes full use of the advantages of rate distortion cost to select the candidate pattern quick descending law. Experiment proves that the proposed algorithm will reduce the number of candidate patterns from 35 to less than 10 compared with RMD, and effectively reduces the complexity of intra-coding. The average encode speed is accelerated about 32%, while BD-Rate loss was only 0.096% encoding time, and subjective image quality has no obvious change after encoding.