Three common numerical optimization algorithms are first studied, including conjugate gradient algorithm, quasi-newton algorithm and LM algorithm. The three kinds of algorithms are used to improve BP neural network respectively and the corresponding classification models based on BP neural network are established. Then the models are used in pattern classification of two-dimensional vectors, and their generalization abilities are also tested. The classification results of different classification models based on BP network are compared with each other. Simulation results show that for small or medium scale networks, BP neural network improved by LM algorithm has the most precise classification result, the fastest convergence speed and the best classification ability. The one improved by conjugate gradient algorithm has the biggest error, slowest convergence speed and worst classification ability. While the classification precision, convergence speed and classification ability of quasi-newton algorithm lie between the above two algorithms.