As an effective way to find correspondences between images, Belief Propagation (BP) is widely used for estimating optical flow in recent years. Nevertheless, its application to directly estimating high-accuracy large displacement optical flow needs huge label space and long time to process. In order to overcome the drawback of BP, we propose a Hierarchical Belief Propagation (HBP) algorithm to estimate high-accuracy large displacement optical flow. We treat input images as Markov Random Fields (MRFs). To accelerate computation, we perform BP on hierarchical MRFs, i.e., superpixel MRF and pixel MRF. The basic displacements obtained on the superpixel MRF are used as a coarse reference to constrain label space to a smaller size on the pixel MRF. Based on this constrained label space, we can estimate accurate optical flow efficiently. Experiments on MPI Sintel dataset show that the proposed method is competitive on speed and accuracy.