Object Detection and Segmentation of Power Equipment in Infrared Images via Improved YOLOv8 and Prompt-Optimized SAM

To achieve automated infrared monitoring of power equipment in substations, this paper proposes an object detection and segmentation method based on improved YOLOv8 and Prompt-Optimized SAM (Segment Anything Model). Firstly, to address the issues of poor resolution and strong background interference in infrared images, the small object feature extraction capability and bounding box regression accuracy of YOLOv8 are improved by introducing a multi-scale feature extraction module, a robust feature downsampling module, and an improved loss function. The Spatial Pyramid Pooling Fast module is improved using large-kernel depthwise separable convolution, enhancing the extraction capability for both global and local features. Secondly, to improve segmentation accuracy, this paper proposes a method that converts detection boxes into prompt points. GrabCut, combined with colour saliency and a superpixel algorithm, is used to segment high-confidence target regions. Zero-shot prompt point segmentation for SAM is achieved by performing clustering on the regions. Experimental validation on an infrared dataset covering seven types of power equipment shows that the improved object detection model achieves an mAP@0.5 of 95.7%, which is 2.3% higher than the original model, with a detection speed of 107.5 FPS. The proposed segmentation method achieves higher accuracy in complex backgrounds than both bounding box-prompted SAM and GrabCut. This study lays a foundation for the precise processing of infrared images of substation power equipment.