Research — Perit

Benchmarks

Two tracks: the words, and their timing.

Speech-to-text leaderboard Word alignment leaderboard

PERIT-STT22 models

Transcription is solved on clean audio. Nobody calls from clean audio.

PERIT-STT scores models on real support recordings — mobile lines, crosstalk, accents and the domain vocabulary that decides whether a transcript is usable downstream.

Latest run · English Word error rate (%, lower is better)

  1. 1 qwen3-asr-flash-2026-02-10 1.53 1.33–1.76
  2. 2 universal-3-5-pro 1.55 1.36–1.78
  3. 3 mai-transcribe 1.51 1.57 1.37–1.80
  4. 4 transcribe-1 1.59 1.39–1.83
  5. 5 qwen3-asr-1.7b 1.60 1.40–1.84

22 models ranked · 24 Sept 2026 More info

PERIT-ALIGN14 models

The right word, a quarter-second late, is still the wrong caption.

PERIT-ALIGN scores the word-level timestamps a speech model returns — where each word starts and where it ends — against reference timings, so captions, redaction and audio search land on the right moment.

Latest run · English Timing accuracy @100 ms (%)

  1. 1 Perit Golden Data Fine-tuned Model 97.85
  2. 2 transcribe-1 92.33
  3. 3 voxtral-mini-transcribe 85.04
  4. 4 grok-stt-1.0 78.04
  5. 5 qwen3-asr-0.6b 77.41

14 models ranked · 24 Sept 2026 More info

Why these two

The right words are only half the job.

01
Speech-to-text fails quietly

A model drops a digit in an account number or mishears a drug name, the transcript still reads well, and the error only surfaces three systems downstream. So the transcription track reports what kind of error a model makes, not just how many.

02
Timestamps fail invisibly

A transcript can be word-perfect and still put every caption a quarter-second late, stop a redaction short of the account number, or send a search hit to the wrong second of the call. The alignment track scores where each word starts and ends, not only what it says.

Where we differ

Three ways speech usually gets measured, and why we do it differently.

None of these are wrong. They answer a different question from the one a team ships against.

  1. Simulated conversations
    Usually: An agent is put through thousands of generated calls and scored by another model.
    Here: Cheap and repeatable, and it inherits whatever the generator thinks a hard call is. We put models on recordings of calls that actually happened, and on live calls with people trained to derail them.

  2. Crowd preference ratings
    Usually: Listeners rank clips they have no stake in — naturalness, warmth, which voice they preferred.
    Here: That measures whether a voice is pleasant, not whether the account got unlocked. Our graders worked the desk the call came from, and score against a written rubric.

  3. Read-speech corpora
    Usually: Clean studio audio, one speaker, scripted sentences, published years ago and long since trained on.
    Here: Held-out support audio: mobile lines, crosstalk, room noise, and the vocabulary that decides whether the transaction goes through.

Behind the boards

How a run is produced, and the notes written along the way.

Method

Six rules every benchmark follows: held-out audio, two-pass references, runs not a run, a human control on every board, operator-written rubrics, provenance on every item.
What we publish, and what we will not

Field notes

Short notes on the decisions that change what a score means, written while building the benchmarks rather than after.
4 notes · method and position

Put a model on the board.

Pick a track. We run your model on the same audio, under the same scoring as every model already on the board, and publish it next to them.

Enter the next run Request the development split