Depth · Physical Intelligence — Perit

What it is

What gets captured, and why it is worth having.

Environment: Living room, afternoon light
Captured with: Stereo passthrough headset, RGB + aligned depth
Task: Pack a mug into a box and close the flaps

The same tasks and rooms as egocentric capture, with a second stream: a per-frame depth map aligned to the RGB image, metric where the sensor supports it. Hand–object distance, container interiors, the gap between a plate and the shelf it is going onto — the geometry a monocular camera has to guess.

Rigs range from a consumer stereo headset to a LiDAR phone to a calibrated stereo pair; calibration files ship with every episode so the depth can be trusted downstream.

What it trains

The spec

How it is captured, and what you receive.

How it is captured

What you receive

Environments and tasks

Where it is recorded, and what people are asked to do.

Tasks are recorded where they actually happen — not in a studio dressed to look like a kitchen. Hover a room to see the kind of setting the protocol calls for.

Kitchens

Living rooms and bedrooms

Offices

Retail and small shops

Warehouses and light industrial

Workshops and labs

Agriculture and outdoor

Client-defined and confidential

What ships

Deliverables, and the labels you can add.

Deliverables

Annotation add-ons

Labelled on the same bench as our speech work, to a written guide, behind the same calibration test.

How a pilot runs

Ten to thirty episodes, then the volume.

  1. 01 day 0
    Brief: The task family, the environments, the rig, the sensors, the episode count. One page, agreed before anything is bought or anyone is recruited.

  2. 02 week 1
    SOP and kit: A written capture protocol — framing, lighting, where a task starts and ends, what counts as a failed take — and the kit list. Operators are trained and tested on it before a single episode is paid for.

  3. 03 week 2
    Pilot episodes: A small batch, ten to thirty episodes, delivered with QC status and a quality report. This is where the protocol breaks — on purpose, and cheaply.

  4. 04 ongoing
    Review and scale: You review the pilot, we fix the protocol, then the run scales on a weekly delivery cadence with the same report attached to every batch.

Turnaround

Before an episode counts

What QC checks, every time.

Samples and the other lines

3 depth episodes to watch and download.

Each with the rig, the room, the task and the file list of the full download.