From Foundation to Field: LISA Fine-Tuning for Mine Open-Vocabulary Segmentation

Underground mining environments are characterized by dim illumination, cluttered man-made structures, and frequent occlusions. However, most existing underground segmentation methods still follow closed-set pixel classification over fixed labels, making them unable to use natural-language instructions or dynamically segment context-specific targets. In this work, we present MineLISA, an instruction-guided segmentation framework adapted from the Language Instructed Segmentation Assistant (LISA) for industrial underground mining applications. MineLISA takes natural-language prompts as input and generates pixel-level masks for underground mining objects on the MUSeg multimodal semantic-segmentation dataset. To adapt LISA under realistic resource constraints, we employ LoRA-based parameter-efficient fine-tuning on vision-language alignment modules and the lightweight segmentation decoder, and re-weight the segmentation loss to emphasize thin, safety-critical structures such as cables and pipelines. This design improves the alignment between textual instructions and underground visual patterns while remaining suitable for hardware with limited GPU memory. Experiments on MUSeg show that, compared with the original LISA, MineLISA achieves substantially improved instruction-conditioned mask predictions and more stable segmentation across diverse underground object categories, indicating its potential for real-world coal-mine deployment.