## Benchmarks

### Two tracks: the words, and their timing.

[Speech-to-text leaderboard](/content/research/benchmark/speech-to-text/leaderboard/index.html) [Word alignment leaderboard](/content/research/benchmark/word-alignment/leaderboard/index.html)

### PERIT-STT22 models

Transcription is solved on clean audio. Nobody calls from clean audio.

PERIT-STT scores models on real support recordings — mobile lines, crosstalk, accents and the domain vocabulary that decides whether a transcript is usable downstream.

Latest run · English Word error rate (%, lower is better)

1. 1 qwen3-asr-flash-2026-02-10 1.53 1.33–1.76  
2. 2 universal-3-5-pro 1.55 1.36–1.78  
3. 3 mai-transcribe 1.51 1.57 1.37–1.80  
4. 4 transcribe-1 1.59 1.39–1.83  
5. 5 qwen3-asr-1.7b 1.60 1.40–1.84

22 models ranked · 24 Sept 2026 [More info](/content/research/benchmark/speech-to-text/index.html)

### PERIT-ALIGN14 models

The right word, a quarter-second late, is still the wrong caption.

PERIT-ALIGN scores the word-level timestamps a speech model returns — where each word starts and where it ends — against reference timings, so captions, redaction and audio search land on the right moment.

Latest run · English Timing accuracy @100 ms (%)

1. 1 Perit Golden Data Fine-tuned Model 97.85  
2. 2 transcribe-1 92.33  
3. 3 voxtral-mini-transcribe 85.04  
4. 4 grok-stt-1.0 78.04  
5. 5 qwen3-asr-0.6b 77.41

14 models ranked · 24 Sept 2026 [More info](/content/research/benchmark/word-alignment/index.html)

### Why these two

The right words are only half the job.

01  
Speech-to-text fails quietly

A model drops a digit in an account number or mishears a drug name, the transcript still reads well, and the error only surfaces three systems downstream. So the transcription track reports what kind of error a model makes, not just how many.

02  
Timestamps fail invisibly

A transcript can be word-perfect and still put every caption a quarter-second late, stop a redaction short of the account number, or send a search hit to the wrong second of the call. The alignment track scores where each word starts and ends, not only what it says.

### Where we differ

## Three ways speech usually gets measured, and why we do it differently.

None of these are wrong. They answer a different question from the one a team ships against.

1. **Simulated conversations**  
   Usually: An agent is put through thousands of generated calls and scored by another model.  
   Here: Cheap and repeatable, and it inherits whatever the generator thinks a hard call is. We put models on recordings of calls that actually happened, and on live calls with people trained to derail them.

2. **Crowd preference ratings**  
   Usually: Listeners rank clips they have no stake in — naturalness, warmth, which voice they preferred.  
   Here: That measures whether a voice is pleasant, not whether the account got unlocked. Our graders worked the desk the call came from, and score against a written rubric.

3. **Read-speech corpora**  
   Usually: Clean studio audio, one speaker, scripted sentences, published years ago and long since trained on.  
   Here: Held-out support audio: mobile lines, crosstalk, room noise, and the vocabulary that decides whether the transaction goes through.

### Behind the boards

#### How a run is produced, and the notes written along the way.

**Method**

Six rules every benchmark follows: held-out audio, two-pass references, runs not a run, a human control on every board, operator-written rubrics, provenance on every item.  
[What we publish, and what we will not](/content/research/method/index.html)

**Field notes**

Short notes on the decisions that change what a score means, written while building the benchmarks rather than after.  
[4 notes · method and position](/content/research/notes/index.html)

### Put a model on the board.

Pick a track. We run your model on the same audio, under the same scoring as every model already on the board, and publish it next to them.

[Enter the next run](/content/contact?topic=Benchmark%20run%20entry/index.html) [Request the development split](/content/contact?topic=Development%20split%20request/index.html)
