In the task of autonomous obstacle avoidance for UAVs, traditional reinforcement learning algorithms face challenges such as high-dimensional state spaces, sparse information, and low exploration efficiency. Although the existing soft Actor-Critic (SAC) algorithm demonstrates strong stability and sample efficiency, its performance in complex environments remains inadequate. To address these issues, this study proposes an improved SAC algorithm, incorporating a squeeze-and-excitation (SE) attention mechanism and random network distillation (RND) module, to enhance the obstacle avoidance capability of UAVs in three-dimensional terrain environments. The SE attention mechanism adaptively adjusts the channel weights of feature maps, enhancing the model’s focus on Critical information and improving feature representation. Meanwhile, the improved RND network promotes the exploration of new environments by generating adversarial targets, thus increasing sample diversity and collection efficiency. Based on the integration of SE and RND, an enhanced feature representation and exploration SAC (EFRE-SAC) framework is constructed, enabling more effective learning of environmental features from depth images and rapid adaptation in three-dimensional environments. Experimental results on the AirSim+UE4 simulation platform demonstrate that the proposed method significantly improves the obstacle avoidance success rate and training efficiency of UAVs, validating the effectiveness of the improved SE and RND modules in reinforcement learning tasks.