Optimal Traffic Allocation Under Heterogeneous Variant Cost

Standard 50/50 traffic splits in A/B testing are inefficient when the treatment variant is more expensive to serve than the control. The optimal allocation is not a simple inverse of the cost, but rather follows a specific mathematical principle. The ideal sample ratio between the treatment and control groups is the square root of the inverse ratio of their marginal costs. This approach correctly balances the cost of adding a subject against the diminishing returns in statistical precision that each new subject provides. This cost-optimal allocation is particularly relevant for experiments involving expensive LLM API calls, discounts, or other costly treatments.