Aiming at the problem of encoder failure of dual rudder wheel AGV under rough road conditions, a scheme of using low-cost RGB-D camera as visual odometer is proposed. This method can avoid the direct kinematics modeling of dual rudder wheel AGV and solve the problem of excessive accumulated error in odometer track estimation. In this study, we use pinhole camera model to find the correspondence between spatial points and pixels firstly. Then the ORB operator is used to extract and match the image, after that the method of Iterative Closest Point(ICP) is used for pose estimation. A visual odometer is built on the platform of Linux + ROS, which is fused with lidar for localization using particle filter algorithm. The positioning effects of encoder and visual odometer are compared in different environments, and the robustness of the whole system is verified.