I Replaced GPT-4 with a Local SLM and My CI/CD Pipeline Stopped Failing

Using GPT-4 for a data extraction task within a CI/CD pipeline resulted in frequent failures due to the model's non-deterministic nature and inconsistent JSON formatting. Despite extensive prompt engineering, cleanup parsers, and using OpenAI's function calling feature, the pipeline remained unreliable because of subtle output variations. The solution involved replacing the large, cloud-based model with a smaller, local language model (SLM) like Qwen2.5-7B. This switch to an SLM provided perfectly deterministic and consistent outputs, successfully stabilizing the pipeline and demonstrating that reliability can be more critical than raw capability for certain production tasks.