Research — Perit
Benchmarks
Two tracks: the words, and their timing.
Speech-to-text leaderboard Word alignment leaderboard
PERIT-STT22 models
Transcription is solved on clean audio. Nobody calls from clean audio.
PERIT-STT scores models on real support recordings — mobile lines, crosstalk, accents and the domain vocabulary that decides whether a transcript is usable downstream.
Latest run · English Word error rate (%, lower is better)
- 1 qwen3-asr-flash-2026-02-10 1.53 1.33–1.76
- 2 universal-3-5-pro 1.55 1.36–1.78
- 3 mai-transcribe 1.51 1.57 1.37–1.80
- 4 transcribe-1 1.59 1.39–1.83
- 5 qwen3-asr-1.7b 1.60 1.40–1.84
22 models ranked · 24 Sept 2026 More info
PERIT-ALIGN14 models
The right word, a quarter-second late, is still the wrong caption.
PERIT-ALIGN scores the word-level timestamps a speech model returns — where each word starts and where it ends — against reference timings, so captions, redaction and audio search land on the right moment.
Latest run · English Timing accuracy @100 ms (%)
- 1 Perit Golden Data Fine-tuned Model 97.85
- 2 transcribe-1 92.33
- 3 voxtral-mini-transcribe 85.04
- 4 grok-stt-1.0 78.04
- 5 qwen3-asr-0.6b 77.41
14 models ranked · 24 Sept 2026 More info
Why these two
The right words are only half the job.
01
Speech-to-text fails quietly
A model drops a digit in an account number or mishears a drug name, the transcript still reads well, and the error only surfaces three systems downstream. So the transcription track reports what kind of error a model makes, not just how many.
02
Timestamps fail invisibly
A transcript can be word-perfect and still put every caption a quarter-second late, stop a redaction short of the account number, or send a search hit to the wrong second of the call. The alignment track scores where each word starts and ends, not only what it says.
Where we differ
Three ways speech usually gets measured, and why we do it differently.
None of these are wrong. They answer a different question from the one a team ships against.
Simulated conversations
Usually: An agent is put through thousands of generated calls and scored by another model.
Here: Cheap and repeatable, and it inherits whatever the generator thinks a hard call is. We put models on recordings of calls that actually happened, and on live calls with people trained to derail them.Crowd preference ratings
Usually: Listeners rank clips they have no stake in — naturalness, warmth, which voice they preferred.
Here: That measures whether a voice is pleasant, not whether the account got unlocked. Our graders worked the desk the call came from, and score against a written rubric.Read-speech corpora
Usually: Clean studio audio, one speaker, scripted sentences, published years ago and long since trained on.
Here: Held-out support audio: mobile lines, crosstalk, room noise, and the vocabulary that decides whether the transaction goes through.
Behind the boards
How a run is produced, and the notes written along the way.
Method
Six rules every benchmark follows: held-out audio, two-pass references, runs not a run, a human control on every board, operator-written rubrics, provenance on every item.
What we publish, and what we will not
Field notes
Short notes on the decisions that change what a score means, written while building the benchmarks rather than after.
4 notes · method and position
Put a model on the board.
Pick a track. We run your model on the same audio, under the same scoring as every model already on the board, and publish it next to them.