Libérer le potentiel des modèles vision-langage sur l'imagerie satellite grâce au fine-tuning

Fine-tuning foundation models can significantly improve performance on specialized tasks by adapting pre-trained models to specific domains. A case study demonstrates this by fine-tuning Mistral's Pixtral-12B, a vision-language model, on the Aerial Image Dataset for satellite image classification. Using Low-Rank Adaptation (LoRA), the model was adapted to better understand the nuanced and ambiguous classes within satellite imagery, a task the base model struggled with. This domain-specific fine-tuning led to a substantial improvement, increasing classification accuracy from 0.56 to 0.91 and reducing model hallucinations from 5% to 0.1%.