GPU Time-Slicing for Concurrent LLM Agents on Kubernetes

Sharing a single GPU between multiple AI agents using Kubernetes and CUDA time-slicing introduces significant hidden performance costs. An experiment co-locating a latency-sensitive agent and a compute-heavy agent on one GPU demonstrates this issue. While Kubernetes reports both pods as healthy and average throughput remains stable, the tail latency (p99) of the latency-sensitive agent increases dramatically. This degradation, invisible to standard dashboards, shows that resource contention can cause critical, time-sensitive agents to fail silently in a production environment. The analysis provides a framework for measuring these microarchitectural costs to reveal the true impact of GPU sharing.