Sparse Matrix-vector multiplication (SpMV) is one of the most frequently used kernels in engineering practice and scientific computing. With the growth of the scale matrix, a large number of calculations restrict the performance of system, so SpMV can be accelerated by utilizing the high computing power of GPU. In this paper, the problem of existing SpMV on GPU is analyzed. Besides, row partition optimization and float4 optimization are designed. Experimental results demonstrate that the proposed approach can enhance the performance by 2-8 times.