Recently, sparse representation has made outstanding contributions to signal and image processing, target recognition and blind source separation. Firstly, the history and rationale of sparse representation are reviewed to summarize the existing sparse representation optimization algorithms. Secondly, those algorithms are divided into the greedy algorithm, the constraint algorithm and other algorithms. The basic principles and characteristics of the first two categories are elaborated respectively, and their representative algorithms and state-of-the-art applications are summarized as well. In addition, the five representative algorithms of the greedy algorithm are simply compared. Finally, the applications of sparse representation in various fields and its general outlook regarding existing problems are offered.