A transition-based rapidly-exploring random tree (T-RRT) algorithm can quickly find a low-risk path for a robot in a two-dimensional complex cost space, but it delivers a poor planning result for an unmanned aerial vehicle (UAV) in the three-dimensional flight condition. To solve this problem, this study proposes an exploring heuristic transition-based RRT (EHT-RRT) algorithm. The algorithm introduces the heuristic idea of the A* algorithm on the basis of the T-RRT to explore the heuristic cost, and it estimates the path cost from the perspectives of risk degree, path length, path deflection angle, and height change to improve the quality of the path. Then, the local node slip strategy is employed to make the path deviate to the low-risk area, and the local best direction attribute is added to each node. At last, the tree node exploration mechanism is improved through three directional vectors, i.e., random direction, target direction, and local best direction, to get rid of the blindness of the T-RRT algorithm in path finding. In addition, a target point offset with a probability of 20% is used to improve the planning efficiency. The results of simulation experiments show that compared with T-RRT, BT-RRT, and T-RRT* algorithms with the same target point offset each, the EHT-RRT algorithm can generate a shorter, safer, and smoother 3D path and better solve the 3D path planning problem of UAV in complex urban environments.