Field notes — Perit
Research Notes
Why the score is task success, not latency
Latency is the easiest speech number to publish and the easiest to game. It belongs in the report, not at the top of it.
A rubric a stranger can apply
If two reviewers grading the same call disagree, the criterion is broken — not the reviewer.
What held-out means when the data is audio
Text benchmarks leak by being quoted. Audio leaks by being reused — the same speaker, the same room, the same script.
The human control is not a ceiling
Publishing a human line at 100 makes the benchmark useless the moment a model gets close.