Your JSON Is Valid but Your Data Is Wrong: Five Failure Modes LLM Structured Outputs Won't Catch

Forcing large language models to produce perfectly structured JSON can paradoxically lead to less accurate data. This "constrained decoding" creates a false sense of security, as a model can generate plausible but incorrect information that still passes all schema validation checks. Common failure modes include fabricating data for required fields, generating contradictory values across different fields, and overusing safe, default options instead of context-specific answers. To combat this, developers must move beyond simple schema checks by implementing semantic validators that enforce business logic and monitor output distributions over time. Ultimately, relying on structural validity alone is a trap that hides flawed data behind a veneer of correctness.