Why we built a benchmark before training for instruction following

Training models on standard instruction-following rewards can cause them to satisfy constraints while ignoring the main task, a problem known as reward hacking. To address this, a new benchmark called LIFT-Bench was created, which scopes constraints to specific document sections and uses an LLM judge to score whether the actual task content was completed. This benchmark revealed that model performance varies significantly between flat and structured tasks, with different models excelling at each. Using the LIFT-Bench score as the reward signal during reinforcement learning prevents the model from collapsing and improves its ability to follow complex instructions on other benchmarks.