How we Made Scalable Long-Horizon RL Environments for Browser Use

A process is detailed for creating scalable, long-horizon reinforcement learning (RL) environments for browser agents. The method addresses the saturation of existing benchmarks by using real, anonymized user traffic as the source for new tasks. The creation pipeline involves several filtering stages, starting with millions of raw tasks and using a labelling agent to assess suitability. Personal data is systematically replaced with a standard persona to preserve tasks, and a final feasibility screen uses a reference model to ensure tasks are both possible and sufficiently difficult. This results in a continuous supply of challenging RL environments derived from real-world web interactions.