AVTensor: A High-Performance Rust Media Decoder for Training Pipelines

Training audio-visual AI models requires perfectly synchronized audio and video, but standard media decoders can fail silently, causing misalignment that harms model performance. After discovering that separate decoders for audio and video led to desynchronized training data, a new tool called AVTensor was developed. AVTensor is an open-source Rust media decoder that uses FFmpeg to process audio and video streams in a single pass, ensuring they are synchronized by construction. The tool also improves efficiency by streaming directly from cloud object storage, which resulted in a 1.8 percentage point increase in model training MFU (Model Flops Utilization).