To improve the precision of equipment defect detection of substations on the premise of only a small number of labeled samples used, this study proposes an improved defect detection algorithm based on the self-supervised model SimSiam. Unlike the original SimSiam, the improved algorithm directly utilizes non-iconic images, such as those in the dataset COCO, rather than using iconic images, like those in the dataset ImageNet, for contrastive learning and achieves performance comparable to that of any supervised methods in downstream defect detection tasks. By replacing multi-layer perception (MLP) networks with fully convolutional networks and spatial attention modules in the projection and prediction heads, the proposed algorithm preserves the spatial structure and local information of high-dimensional features. Furthermore, the output feature map is mean-pooled before similarity is calculated to obtain the eigenvector, which is then normalized to calculate the Euclidean distance and further modify the loss function of self-supervised contrastive learning. The experimental results show that the improved algorithm can make full use of non-iconic images for contrastive learning and improve the precision of equipment defect detection of substations on the premise of labeling only a small number of samples. Its mean average precision (mAP) reaches 83.84% in the detection of five types of defects, namely, broken meter dials, hanging suspended substances, nests, respirator silicone discoloration, and abnormal closure of the box door.