0
Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL
https://huggingface.co/blog/asyncgrpo-lora-hfjobs(huggingface.co)TRL's `AsyncGRPOTrainer` now supports training LoRA adapters, enabling a distributed training setup across separate Hugging Face Jobs. This architecture decouples the trainer from the vLLM inference replicas, eliminating the need for them to share a machine or use NCCL for synchronization. Instead, the small LoRA adapter is shared between jobs using a Hugging Face Storage Bucket mounted as a common filesystem. A proxy server manages routing rollouts and broadcasting adapter updates to the vLLM replicas, with the system's performance being iteratively improved by analyzing metrics to identify and resolve bottlenecks.
0 points•by chrisf•1 hour ago