Short text matching is a core problem in the field of natural language processing, which can be applied to tasks such as information retrieval, question answering systems, and question paraphrase. Most of the past work only considered the internal information of the text when extracting text features, ignoring the interactive information between two texts, or only performed single-level interaction. Given the above problems, an Improved Short Text Matching model (ISTM) based on Transformer is constructed. The ISTM model takes DSSM as the basic architecture and uses the BERT model to vectorize the text to solve the ambiguity of Word2Vec. It relies on the Transformer encoder to extract features of the text and obtain its internal information. It considers the multi-level interactive information between the two texts and finally infers and computes the degree of semantic matching between two texts by the concatenated vector. Experiments show that compared with the classic deep short text matching model, the ISTM model proposed in this study shows better results on the LCQMC Chinese dataset.