Vision Language Model Alignment in TRL ⚡️

New alignment methods for Vision Language Models (VLMs) have been added to the TRL library, offering advanced alternatives to Direct Preference Optimization (DPO). The techniques include Mixed Preference Optimization (MPO), which uses a combined loss to improve reasoning, and Group Relative Policy Optimization (GRPO), which updates policies over groups of responses for greater robustness. These methods are designed to extract more signal from preference data and scale better with modern VLMs. The integration includes training scripts and code examples for implementing these new alignment strategies.