Google Cloud C4 Brings a 70% TCO improvement on GPT OSS with Intel and Hugging Face

Intel and Hugging Face collaborated to benchmark an open-source GPT Mixture of Experts (MoE) model on Google Cloud's latest C4 Virtual Machines. These new C4 instances, powered by Intel Xeon 6 processors, were compared against the previous generation C3 instances running 4th Gen Intel Xeon processors. The results demonstrated a significant 1.7x improvement in Total Cost of Ownership (TCO) for the C4 VMs when running the LLM. The benchmark also showed a 1.4x to 1.7x increase in throughput per vCPU per dollar, highlighting the performance and cost-efficiency gains of the new hardware for CPU-based inference.