Given the ORB algorithm has no scale invariance and exists more false match in image keypoints matching, this paper combines SURF and bilateral matching algorithm to improve ORB. The improved ORB algorithm is named SSORB(scale invariance and symmetrical matching ORB). First, we generate multi-scale space by different sizes of box filter template convoluting with integral image. Second, we detect the stable extreme points in multi-space, making the extracted points invariant to scale. Third, we describe these points with ORB descriptor making them invariant to rotation. At last, according to the Hamming distance match these points. Due to the existence of error match, bilateral matching is used to improve the match accuracy by eliminating the false match. The experimental results show that the SSORB algorithm effectively solves the scale invariant defects of the ORB algorithm, meanwhile, keeps the speed advantage of ORB and improves the matching accuracy.