Due to powerful self-learning ability, reinforcement learning has become a research hot spot about robot navigation problems, but the operating efficiency and convergence speed of the algorithm are tried by the the complex unknown environment. A new Q-learning algorithm for robot navigation was proposed in this paper. First, three discrete variables were used to define the space states of the environment, and then two parts of the reward functions were designed, combining the beneficial knowledge for reaching the target to inspire and guide the robot's learning process. The experiment was executed on Simbad simulation platform. The results show that the proposed algorithm is well done in an unknown environment robot navigation task, and has a better convergence speed.