title: "Comment les caches KV affectent le délai avant le premier token pour les LLMs"

KV caching is a crucial technique for optimizing the performance of Large Language Models. This method specifically targets the reduction of "time to first token" (TTFT) latency, which measures how quickly a model begins to generate its response. By caching key-value pairs from the input prompt, models can avoid redundant computations and deliver the first part of an answer much faster. This optimization not only improves the user's perceived speed but also enhances the overall throughput and efficiency for handling subsequent calls.