Field notes — Perit

Research Notes

Why the score is task success, not latency

Latency is the easiest speech number to publish and the easiest to game. It belongs in the report, not at the top of it.

Read the note

A rubric a stranger can apply

If two reviewers grading the same call disagree, the criterion is broken — not the reviewer.

Read the note

What held-out means when the data is audio

Text benchmarks leak by being quoted. Audio leaks by being reused — the same speaker, the same room, the same script.

Read the note

The human control is not a ceiling

Publishing a human line at 100 makes the benchmark useless the moment a model gets close.

Read the note