Although the SemBERT model is an improved version of the BERT model, it has two obvious defects. One is its poor ability to obtain vector representation. The other is that it directly uses conventional features to classify tasks without considering the category of the features. A new feature reorganization network is proposed to address these two defects. This model adds a self-attention mechanism into the SemBERT model and obtains better vector representation with an external feature reorganization mechanism. Feature weights are also reassigned. Experimental data show that the F1 score of the new method on the Microsoft Research Paraphrase Corpus (MRPC) dataset is one percentage point higher than that of the classical SemBERT model. The proposed model has significantly improved performance on small datasets, and it outperforms most of the current outstanding models.