Respiratory rate is one of the important indicators of human health. To solve the problems of the existing respiratory rate detection methods including one single human posture and poor detection accuracy and robustness, this study proposes a visual detection method of human respiratory rate suitable for multiple postures. This method uses an ordinary camera to capture human breathing videos. The image pyramid optical flow is used to process continuous video images and thereby obtain the moving foreground region, wherein the largest connected area is preliminarily identified as the thoracoabdominal breathing area. Then, the breathing region in each frame of the video is input into the complex steerable pyramid for multi-scale and multi-directional spatial decomposition, and amplitude spectra and phase spectra on multiple scales and in multiple directions are obtained. On this basis, the phase spectra on multiple scales and in multiple directions of each frame are weighted by the amplitude spectra and then averaged to obtain the phase-time signal. Finally, decisions are made for the extracted signal. If the dominant frequency of the signal is within the frequency band of the respiratory signal and the energy proportion is high, the respiratory rate is obtained by peak detection of the signal. Otherwise, continuous video images are reselected for subsequent detection. The experimental results show that this method is suitable for respiratory rate detection in various postures and that it is superior to the existing methods in accuracy and robustness.