Post-training case study: Reaching frontier-level attack resilience with Alice

AI21 Labs post-trained the NVIDIA Nemotron 3.5 Super VL model to significantly improve its resilience to attacks like indirect prompt injection. The resulting model demonstrated a 41-point improvement over the base model and surpassed the performance of GPT-6.1 Sol at a 25x lower cost. The training process involved a supervised fine-tuning (SFT) stage using rephrased data to prepare the model for reinforcement learning. This was followed by online reinforcement learning (RL) using a modified GRPO method within simulated environments to enhance security without degrading the model's ability to complete its tasks.