Deep reinforcement learning can be used to extract effective information from high-dimensional images and thus automatically generate effective strategies for solving complex tasks such as game AI, robot control, and autonomous driving. However, due to the complexity of the task environment and the low exploration efficiency of the agent, it is still necessary for the agent to interact with the environment frequently even for relatively simple tasks. Therefore, this study proposes a CCLF algorithm (Bootstrapped CCLF), which combines Bootstrapped exploration method to generate more different potential actions through multiple heads in the actor network, so that more different states can be accessed to improve the exploration efficiency of the agent, and thus the convergence process can be accelerated. The experimental results show that the algorithm has better performance and stability than the original algorithm in the DeepMind Control environment, which proves the effectiveness of the algorithm.