The shaking of Unmanned Aerial Vehicles (UAVs) is an important reason for the error caused by the visual sensors in extracting vital signs. To solve this problem, this paper proposes a method of respiration rate detection based on Variational Mode Decomposition (VMD), which is immune to UAV shaking. First, a complex controllable pyramid is used to extract the initial characteristics of respiration rates. Second, an extraction method of respiratory signals based on VMD is designed to obtain the candidate respiratory modal signals. Third, the eigenmodes with the minimum variance are selected for respiration rate detection. The experimental results show that the proposed method can effectively extract the intrinsic respiratory signals under the normal shaking condition of the UAVs. Moreover, this method can detect respiration rates in different human postures at different measured distances, and its detection accuracy is higher than that of the existing methods.