Automatic text-speech alignment technology is widely used in speech recognition and synthesis, content production, and other fields. Automatic text-speech alignment aims to align speech with text in sentence, word, and phoneme units and obtain the time alignment information. Most of the recent alignment methods are based on automatic speech recognition (ASR). On the one hand, the alignment accuracy is limited by the word error rate (WER) of ASR. On the other hand, such methods cannot effectively align imperfect transcriptions. This study proposes a text-speech alignment method based on anchor and prosodic information. Through fragment annotation based on boundary anchor weighting, speech is divided into aligned and unaligned fragments. For unaligned fragments, this study extracts their prosodic information by a dual-threshold endpoint detection method and detects the boundaries of sentences. This approach reduces the dependence of ASR-based text-speech alignment on the speech recognition effect. Compared with the current advanced ASR-based text-speech alignment methods, the proposed method can improve alignment accuracy by more than 45% when the WER is 0.52 and by at least 3% when the degree of incomplete matching is 0.5.