To further improve the accuracy of flower classification, this study proposes a network model based on visual attention mechanism after the research on the VGG16 network model. Squeeze-and-excitation (SE) attention is embedded in the VGG16 network model to extract salient region features of flowers. BN layer is added following the convolutional layer to effectively prevent gradient explosion and gradient disappearance and to speed up the training and convergence of the network. Multi-loss function fusion is utilized to train the new model. The new model can effectively extract salient regions such as stamens and petals. It can enlarge the distance between and within classes and accelerate the convergence, further improving the classification accuracy. Experimental results show that the new model advances in the classification accuracy of the Oxford-102 dataset after the introduction of the attention mechanism and outstrips the current reference.