Effective detection of damage and foreign matter on transmission lines is very important for intelligent circuit inspection. However, it is difficult to collect data from different power companies to train a unified detection model due to the data island problem. Therefore, this study proposes a circuit defect detection method based on federated transfer learning by combining federated transfer learning and object detection algorithms. Specifically, a high-performance detection model is selected as the basic detection model, whose initial weight is frozen. The model adaptively learns from the data of different clients by using the low-rank decomposition of the weight matrix and inserting an adapter layer, so as to greatly reduce the number of the trainable parameters. An adaptive weight screening method is also proposed to accurately determine the low-rank decomposition of the weight layer and the insertion position of the adapter layer of the model. Through simple adaptive learning, the model can effectively adapt to the data distributions from different power companies. Experimental verification on a power dataset that closely resembles real-world conditions shows that the proposed model can adapt to different distributed detection scenarios under the premise of ensuring the security and privacy of customer data.