Back
Nov 17, 2026

We Tested 27 TTS Models on Hard Names. Most Readings Came Out Wrong.

TTS leaderboards report word error rates of a few percent. Then you ship a script full of names, and the voice says them wrong.

Podonos, the company behind Onepin, built a benchmark to measure exactly that gap. The Onepin Hard-Word Pronunciation Benchmark tests 27 commercial text-to-speech models on 484 hard words: cities, people, drugs, brands, mountains and mythology. After Onepin's pronunciation check and correction, pronunciation accuracy across the line rose from 90.0% to 98.0%, with every one of the 27 models finishing between 96.6% and 98.5%. On the hard words themselves, accuracy rose from 44.8% to 87.8%.

The data is open. This post covers what we measured, what we found, and where the numbers have limits.

Why don't TTS leaderboards catch mispronounced names?

Most TTS leaderboards score voices in one of two ways.

Human preference. Listeners hear two voices and pick the one they prefer. Persona, pace, tone and naturalness decide the result. A listener who does not know how Quelimane is said cannot tell that the voice said it wrong.

Speech-to-text word error rate. A speech-to-text model transcribes the generated line, and the transcript is compared with the script. This is fast and cheap, but it is only as accurate as the speech-to-text model, and it averages over every word in the line. In a 25-word sentence, one mangled name barely moves the score.

Neither method checks the words that break in production: the names. In a documentary, an audiobook, a training course or a pharmacy line, that is the error the listener remembers.

What does the benchmark measure?

We wrote 100 sentences, each carrying four or five hard words, 484 words in all across 32 categories. Every sentence was voiced by 27 commercial TTS models, the providers integrated in Onepin at the time.

Sentences100
Hard words484 unique, 13,068 readings
TTS models27, from 17 provider catalogues
Clips2,700 (9.5 hours of audio)
Voices1,383 distinct voices from each provider's English catalogue
SettingsProvider defaults, plain-text input, no pronunciation controls
Language and dateUS English, generated 20 September 2026

Every clip was scored twice: once exactly as the provider generated it (the source TTS baseline in every table below), and once after Onepin's pronunciation check and correction had run on it.

The scoring uses PED, Onepin's pronunciation check. PED listens to the audio, writes down each speech sound it hears, and compares those sounds with the accepted pronunciations of each word. Each hard word has a primary reading in IPA and, where legitimate variants exist, further accepted readings, checked against dictionaries, Wikipedia, Wiktionary and official sources.

The benchmark reports two numbers, and they answer different questions:

  • Pronunciation accuracy scores the whole line, from 0 to 100: the share of sounds heard right, minus a deduction for each hard word read wrong. It is the same PED score Onepin shows on every line it delivers.
  • Hard-word accuracy scores only the names: the share of hard-word readings within one wrong, missing or extra sound of an accepted reading for a word of four to seven sounds, with proportionally more allowed for longer words.

How often do commercial TTS models get hard names wrong?

Across all 27 modelsSource TTSWith Onepin
Pronunciation accuracy (whole line)90.0%98.0%
Hard-word accuracy (names only)44.8%87.8%

Read the first two rows together. At 90.0%, source TTS gets most sounds in a line right. The sounds it gets wrong sit almost entirely inside the hard words: source TTS reads more than half of the names wrong, and only 2 sentences in 100 come out with every name right.

Out of every 100 hard words: 45 read right by source TTS, 43 more fixed by Onepin, 12 still wrong

This is the gap line-level scores hide. A model can sound fluent across a sentence and still miss the one word the listener was waiting for.

The spread between models is wide before correction. On pronunciation accuracy, the 27 models range from 85.9% to 92.3%. On hard words, the best model reads 55.6% right and the lowest 28.3%.

The hardest categories are names of volcanoes, rivers, islands, fortresses and religious buildings, read right 27.6% to 35.0% of the time. Cities, the largest category, sit close behind at 36.2%. The easiest are words that appear in English text every day: software, companies, computer-science terms and sports, from 74.1% down to 65.3%.

Hear the difference

Each pair below is the same sentence in the same voice: first as the TTS provider generated it, then after Onepin's pronunciation check and correction. Under each clip is a plain respelling of what PED heard.

JaworzynkaVillage in Poland

“…whose grandmother still lives in the mountain village of Jaworzynka…”

Should be yah-vo-ZHIN-kuh

Source TTSchuh-WOR-zing-kuh
With Onepinyah-voh-ZHING-kuh
MontilletFrench Olympic downhill skier

“…retrospective paired footage of Montillet racing downhill with…”

Should be mohn-tee-YAY

Source TTSMON-tih-let
With Onepinmon-tee-YAY
NagycserkeszVillage in Hungary

“…river and in villages like Nagycserkesz ending with a…”

Should be NAHJ-chair-kess

Source TTSNAH-dick-ser-kuss
With OnepinNAHJ-chair-kuss
TanztheaterGerman dance-theatre form

“…Tanztheater rehearsal the choreographer described…”

Should be TAHNTS-tay-ah-ter

Source TTSTANZ-thee-uh-ter
With OnepinTAHNS-tee-ah-ter
FruelaMedieval king of Asturias

“…His lectures wandered from King Fruela of Asturias…”

Should be FRWAY-luh

Source TTSfroo-EL-uh
With Onepinfroh-WAY-luh
StyrvollWooden church in Norway

“…circle the wooden Styrvoll Church and the distant asteroid…”

Should be STEER-vawl

Source TTSSTUR-vohld
With OnepinSTEER-vawl
GridrTiny moon of Saturn

“…Lake and the tiny Saturnian moon called Gridr…”

Should be GREE-thur

Source TTSKRYDE
With OnepinKREE-thur
ChanyikheiMinor planet

“…and tracks the minor planet Chanyikhei…”

Should be chahn-yik-HAY

Source TTSCHAN-ee-kee
With Onepinchahn-ik-HAY

The pattern is consistent. The model reads the spelling as English, so the German th in Tanztheater turns into an English th, and Montillet ends on a hard t. The voice is fine. The name is wrong.

Excerpts are about three seconds, cut on the same words in both versions and set to the same loudness. All audio is AI-generated speech, and every "With Onepin" clip is the provider's own audio after Onepin's correction. The full clips, with the TTS model behind each, are in the Hugging Face dataset viewer.

How does Onepin correct the names?

Onepin runs the same PED check on every line it generates. For each hard word, it compares the sounds in the audio with the word's accepted readings. When a word misses, Onepin re-voices only that word, in the same voice, splices it back into the line, and checks the line again. The rest of the line stays as the provider generated it.

Across the benchmark:

  • Pronunciation errors in the line fell from 10% to 2%. Pronunciation accuracy rose from 90.0% to 98.0%.
  • Hard-word accuracy rose from 44.8% to 87.8%. 88 names in 100 now come out right.
  • Every one of the 27 models improved. On pronunciation accuracy, the smallest gain was 6.2 points, and all 27 models finished within two points of each other, from 96.6% to 98.5%.

The last point matters for anyone choosing a TTS provider. Before correction, the gap between the best and worst model on hard words was more than 27 points. After correction, the gap on pronunciation accuracy is under two. The choice of TTS still shapes the voice. For the names, the check matters more.

How far can you trust these numbers?

This benchmark was built, run and judged by Podonos, the maker of Onepin. That is a reason to read the limits, so here they are.

  • The source TTS baseline uses provider defaults. Plain-text input, no SSML, no phoneme tags and no custom lexicons. Providers' own pronunciation controls could raise their source scores.
  • One language, one snapshot. US English, generated on 20 September 2026. Version 1.1 adds a blind listening round on this audio and a re-run with one fixed voice per model.
  • Listeners agree with the check on source errors. In a blind listener study, PED caught 9 in 10 of the pronunciation errors listeners heard in source TTS.
  • Two providers are anonymized. Their terms of service restrict publishing benchmark results that name them, so they appear as Provider A and Provider B, with model and voice names withheld.

No provider paid for inclusion, was consulted on the sentence set or saw the results before publication. The full scoring rule, the second judge and the statistics are in METHOD.md.

Run it on your own TTS

Everything needed to check these results is public:

Run your own model on the sentence set and compare. If you think a reading is wrong, open an issue: a changed reading gets a changelog entry and a re-score.

To run the same check and correction on your own script, add the pronunciation check and the Pronunciation Corrector to an Onepin workflow.

Leaderboards measure the voice. Production measures the names.

A leaderboard can tell you which voice people prefer and how often a speech-to-text model agrees with the script. It cannot tell you whether the voice says Jaworzynka, Montillet or Gridr the way the people who live there, or study them, say them.

Onepin is the AI Voice Production Agent. It plans, generates, validates and retries every line before it ships, and the names are where that validation earns its place.

Frequently asked questions

What is the Onepin Hard-Word Pronunciation Benchmark?
It is an open benchmark from Podonos, the company behind Onepin, that measures how well commercial text-to-speech models pronounce hard words: cities, people, drugs, brands, mountains and mythology. It covers 100 sentences with 484 hard words, voiced by 27 commercial TTS models from 17 provider catalogues, for 2,700 clips and 13,068 hard-word readings. The sentences, reference readings and scoring results are public on GitHub and Hugging Face.
How accurate are commercial TTS models on hard names?
Across 27 models, using each provider's default settings and plain text, 44.8% of hard-word readings were within one sound of an accepted pronunciation. Only 2 sentences in 100 had every hard word right. The best single model read 55.6% of hard words right and the lowest read 28.3%.
What is the difference between pronunciation accuracy and hard-word accuracy?
Pronunciation accuracy scores the whole line: the share of speech sounds heard right, minus a deduction for each hard word read wrong. Across the 27 models it was 90.0% before Onepin and 98.0% after. Hard-word accuracy scores only the names inside the line: 44.8% before Onepin and 87.8% after. A line can score well on the first while getting most of its names wrong.
How does Onepin fix mispronounced names?
Onepin checks every hard word in the generated audio against its accepted pronunciations. When a word misses, Onepin re-voices only that word in the same voice and checks the line again. Across the benchmark, pronunciation errors in the line fell from 10% to 2%, and every one of the 27 models improved.
Can I run the benchmark on my own TTS model?
Yes. The sentences, the accepted readings for each hard word and the scoring method are open on GitHub, and per-clip scores and showcase audio are on Hugging Face. You can also run the same pronunciation check and correction on your own script inside an Onepin workflow.

Ready to publish?

Turn any script into production-quality voice,
in any language, in minutes.

Run your first line