With the development of the network, the public data which shows the trend of explosive growth, making the data type more and more complex. These network data combine with each other to form a complex network data structure to express the information of data. In this scenario, it is increasingly difficult to fully express data information through a single type of data (picture, text, voice, etc.). For the purpose of a network information that contains multiple types of data can be classified better, this study proposes a new public opinion classification model via neural network which is used to learn the data features respectively, and to classify their features after fusion. In the experiment, LSTM and CNN neural networks are used to extract text and image's features, fusing the two features to classified. The experimental results show that the reclassification after the fusion of various data features can better realize the classification and improve the accuracy of data information classification.