The generation of text adversarial samples is of great significance for studying the vulnerability of deep learning-based natural language processing (NLP) systems and improving the robustness of such systems. This work studies the important steps in the generation of word-level adversarial samples and the search for replacement words. Considering the problems of premature convergence and poor effectiveness of existing algorithms, a text adversarial sample generation method is proposed, which is based on an improved artificial bee colony (ABC) search algorithm. Firstly, the search space of the words to be replaced is obtained by the screening of the sememe annotations of the words in the HowNet database. Then, the improved ABC algorithm is employed to search and locate the replacement words for the generation of high-quality text adversarial samples. Finally, attack tests are conducted on two text classification datasets for a comparison with the current mainstream text classification models based on deep neural networks (DNNs). The results demonstrate that compared with the existing text adversarial sample generation methods, the proposed method can mislead the text classification system with a higher success rate of attack and preserve semantic and grammatical correctness to a larger extent.