This paper proposes an algorithm for the bidirectional pedestrian counting based on the classification of moving objects in intelligent video surveillance. First of all, detect and track the moving objects. According to the feature information of the blocks of moving objects has been detected through the predetermined counting line, divide the moving objects into non-pedestrian, single pedestrian and multiple pedestrians. For the case of multiple pedestrians, detect the head and shoulder in the blocks using Hog and SVM and count the number of pedestrians. In this paper, with the direction information of moving objects and the number of pedestrians including in the blocks of moving objects, calculate the bidirectional flow of people when the pedestrians cross the the predetermined counting line. The algorithm in this paper acquires good effect in the corridor environment.