Humyn Labs Launches Report for AI Voice Benchmarking Across Global South Languages
BRIDGE, the largest independent benchmarking report, evaluates 15 commercial models across 22 non-English languages using a seven-metric scoring stack
BENGALURU, India, May 13, 2026 /PRNewswire/ -- Humyn Labs, a physical and voice AI data infrastructure company, today published BRIDGE (Benchmark of Regional & International Data for Global Evaluation), the largest independent benchmark for commercial AI speech-recognition tools on real conversational data from non-English languages across the Global South. Testing 15 models across 22 languages, including Latin American Spanish dialects, Brazilian Portuguese, and Vietnamese, the benchmark evaluates languages spoken by over 5.5 billion people. Read the full report here.
The findings reveal that global rankings are not reliable indicators of regional performance. ElevenLabs Scribe v2 leads overall at 10.6% word error rate, but ranks third in Vietnamese, where AssemblyAI Universal performs best despite ranking 12th globally. The report also found major performance gaps across Venezuelan Spanish, Brazilian Portuguese, and cross-regional speaker pairs.
"The models are grading their own work. ASR providers published their own accuracy scores using benchmarks built on English-first, internet-trained datasets, with little independent validation. Meanwhile, enterprises are making million-dollar deployment decisions on numbers that rarely reflect how their users in Global South actually speak. Before BRIDGE, there was no independent benchmark for real-world conversational audio across non-English markets," said Manish Agarwal, Co-founder, Humyn Labs.
BRIDGE applies seven metrics beyond standard word and character error rates, including Semantic Similarity, Code-Switch F1, Loan Word WER, Phoneme-Informed Error Rate, and Word Information Lost. The benchmark found that models with high word error rates in Vietnamese still preserved over 93% meaning accuracy.
"The models aren't the only problem the metrics are. You cannot evaluate non-english speech with a scoring system designed for English phonology and call it rigorous. The performance leaderboard for Spanish is not the leaderboard for Vietnamese. A single aggregate benchmark score cannot support cross-regional deployment decisions," said Ishank Gupta, Co-founder, Humyn Labs.
The benchmark was built on real two-person conversations collected across multiple regions rather than scripted or internet-scraped audio. Each metric reveals a dimension of model quality that headline accuracy figures obscure, and together they show that the tools currently ranked highest globally are not necessarily the right tools for the markets where voice AI adoption is growing the fastest. The full dataset is available on Hugging Face.
Contact Details: Sharmilee Daru | [email protected]
Photo - https://mma.prnewswire.com/media/2977812/Humyn_labs.jpg
SOURCE Humyn Labs
Share this article