Abstract:The development of neural network compression relieves the difficulty of deep neural networks running on resource-restricted devices, such as mobile or embedded devices. However, neural network compression encounters challenges in automation of compression, conflict of the sparsity and hardware deployment, avoidance of retraining compressed networks and other issues. This paper firstly reviews classic neural network models and current compression toolkits. Secondly, this paper summarizes advantages and weaknesses of representative compression methods of parameter pruning, quantization, low-rank factorization and distillation. This paper lists evaluating indicators and common datasets for the performance evaluation and then analyzes compression performance in different tasks and resource constraints. Finally, promising development trends are stated in this paper as references for promoting the neural network compression technique.