0

Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL

https://huggingface.co/blog/asyncgrpo-lora-hfjobs(huggingface.co)
TRL's `AsyncGRPOTrainer` now supports training LoRA adapters, enabling a distributed training setup across separate Hugging Face Jobs. This architecture decouples the trainer from the vLLM inference replicas, eliminating the need for them to share a machine or use NCCL for synchronization. Instead, the small LoRA adapter is shared between jobs using a Hugging Face Storage Bucket mounted as a common filesystem. A proxy server manages routing rollouts and broadcasting adapter updates to the vLLM replicas, with the system's performance being iteratively improved by analyzing metrics to identify and resolve bottlenecks.
0 pointsby chrisf1 hour ago

Comments (0)

No comments yet. Be the first to comment!

Want to join the discussion?