Can Voice Agents Handle Bilingual Customers? Benchmarking Frontier ASR on Code-Switched Speech

A new benchmark evaluates how well Automatic Speech Recognition (ASR) models handle code-switching, where bilingual speakers mix languages within a single conversation. The benchmark focuses on enterprise IT and HR scenarios across four language pairs: Spanish-English, French-English, Canadian French-English, and German-English. Seven ASR models were tested using metrics like Word Error Rate (WER), Semantic Word Error Rate (SWER), and Answer Error Rate (AER) to measure transcription accuracy and meaning preservation. The results indicate that performance varies significantly by model and language, with ElevenLabs Scribe V2, Gemini 3 Flash, and Assembly AI Universal 3-Pro emerging as top performers.