With the extensive application of SIP protocol, the security mechanisms of SIP network have gradually become a hot topic. This article introduces relative entropy into SIP network, and proposes a detection algorithm based on it to against SIP DoS flooding attacks. We also compare this algorithm with the traditional entropy detection algorithm. The experimental results showed that: the entropy and relative entropy are both stable under normal network traffic. But relative entropy fluctuated significantly when attack occurred and our algorithm have a higher detection rate and more accurate results than entropy detection algorithm.