Training Design for Text-to-Image Models: Lessons from Ablations

This document details experiments in training text-to-image models, building upon a previous post about model architecture. It establishes a baseline configuration using a 1.2B parameter model with a pure Flow Matching setup to serve as a reference point. The authors then explore various techniques to improve training efficiency and model performance, including representation alignment methods like REPA and different training objectives such as Contrastive Flow Matching. The piece also covers data strategies, such as using synthetic images, and practical tips like optimizer choice and numerical precision handling.