Real-Time Video at Batch Size One

A 57B-parameter video diffusion transformer powers Runway Characters, enabling real-time conversational video generation at 24 frames per second. To meet the strict 166.7ms per-step latency requirement, significant optimizations were implemented, reducing the number of required H100 GPUs from eight to four. Key improvements included overhauling the attention cache to perform work once at write time and fusing the tensor-parallel all-reduce communication step into the matrix multiplication compute kernel. These changes successfully reduced the diffusion step time to below the real-time threshold without degrading output quality, as measured by task-specific metrics like lip-sync scores.