monday Service + LangSmith: Building a Code-First Evaluation Strategy from Day 1

The monday.com Service team embeds AI agent evaluation directly into their development process, treating it as a foundational requirement from day one. They employ a dual-layered strategy, starting with "offline" evaluations that act as a safety net to test core logic and edge cases against curated datasets. To accelerate development, they parallelized these tests and slashed feedback loop times from nearly three minutes to just 18 seconds. This is complemented by "online" evaluations that continuously monitor the agent's performance on live, multi-turn production conversations to ensure real-world quality and business impact.