Tactile · Physical Intelligence — Perit
What gets captured, and why it is worth having.
Environment: Kitchen counter, daylight
Captured with: Tactile glove, fingertip and palm pressure pads, wrist camera
Task: Pick up soft and fragile objects without damage
Vision tells a policy that a hand touched a cup; it does not say how firmly, or when the grip slipped. The tactile line adds a pressure map per fingertip and palm — or per gripper jaw — synchronised to the video, with contact onset, hold and release marked per event.
Tasks are chosen for contact richness: fragile and deformable objects, lids and fasteners, stacking with feedback, and hand-overs. Every episode ships with the raw pressure stream and the derived contact events, so you can re-threshold it yourself.
What it trains
- Force-aware grasping and in-hand manipulation
- Slip detection and grip adjustment
- Contact prediction from vision
- Tactile–visual representation learning
How it is captured, and what you receive.
How it is captured
- Primary rig: Tactile glove, fingertip and palm pressure pads, both hands
- Gripper: Two-finger gripper with tactile jaw pads, handheld or on a robot
- Cameras: Wrist camera plus head-mounted RGB from the egocentric line
- Force–torque: Wrist sensor on request
- Calibration: Per-glove and per-pad, in newtons, every session
- Sync: Video, tactile and gripper state on one clock
What you receive
- Video: H.264 in MP4, head plus wrist streams
- Tactile: Per-taxel pressure at 100–500 Hz, calibrated to newtons per pad
- Events: Contact onset, hold and release per finger or jaw
- Sync: Common clock across video, tactile and gripper state
- Calibration: Per-glove and per-pad calibration files per session
- Episode: 20 seconds – 4 minutes
Where it is recorded, and what people are asked to do.
Tasks are recorded where they actually happen — not in a studio dressed to look like a kitchen. Hover a room to see the kind of setting the protocol calls for.
Kitchens
- Indian, European and Brazilian home kitchens, lived-in and cluttered, plus small commercial kitchens and pantries.
Cooking preparation
- Washing up
- Cupboards and drawers
- Appliances
- Pouring and scooping
Living rooms and bedrooms
- Real Western homes in Spain, Germany, France and Italy, and apartments in India, Brazil, Japan and Korea. Sofas, wardrobes, shelves, laundry.
Tidying and folding
- Making beds
- Shelving and storage
- Laundry
- Furniture interaction
Offices
- Open-plan and small offices: desks, pantries, meeting rooms, filing and cable clutter.
Desk tasks
- Document handling
- Cables and peripherals
- Pantry tasks
- Drawers and storage
Retail and small shops
- Kirana stores, groceries, pharmacies and supermarket aisles, with real stock and real customers out of focus.
Shelf restocking
- Picking to a list
- Bagging and counters
- Inventory checks
- Crates and boxes
Warehouses and light industrial
- Racking aisles, packing benches, courier depots and small production lines with pallets, trolleys and parts bins.
Pallet to shelf
- Parcel sorting
- Packing and taping
- Parts movement
- Inspection
Workshops and labs
- Mechanical and electronics workshops, repair benches and robotics labs where a follower rig can be set up.
Tool handling
- Assembly and fastening
- Pegboards and racks
- Bench tidying
- Teleoperation setups
Agriculture and outdoor
- Polytunnels, packing sheds, market stalls and yards, in daylight that changes from one episode to the next.
Sorting produce
- Crates and trays
- Pruning and picking
- Weighing and bagging
- Loading
Client-defined and confidential
- Your site, or a space recreated to your drawings, run by an NDA'd crew with limited-visibility workflows and no clip leaving your region.
Proprietary workflows
- Recreated spaces
- Stealth programs
- On-site with your rig
- Your robot, our operators
Deliverables, and the labels you can add.
Deliverables
- Video plus synchronised pressure stream per episode
- Contact event log per finger or jaw
- Task and environment metadata per episode
- QC status and reviewer chain per episode
- Consent reference on every file
- Quality report per batch: accepted, rejected, and why
- Weekly drops to your bucket, region pinned at kickoff
Annotation add-ons
- Object material and weight labels
- Slip and failure annotations
- Grasp type per event
- Hand pose from the hand-pose line
How a pilot runs
Ten to thirty episodes, then the volume.
01day 0: The task family, the environments, the rig, the sensors, the episode count. One page, agreed before anything is bought or anyone is recruited.
02week 1: A written capture protocol — framing, lighting, where a task starts and ends, what counts as a failed take — and the kit list. Operators are trained and tested on it before a single episode is paid for.
03week 2: A small batch, ten to thirty episodes, delivered with QC status and a quality report. This is where the protocol breaks — on purpose, and cheaply.
04ongoing: You review the pilot, we fix the protocol, then the run scales on a weekly delivery cadence with the same report attached to every batch.
What QC checks, every time.
- Framing: The task stays in frame; hands and workspace inside the protocol's bounds for the whole episode.
- Exposure and focus: No blown highlights or focus hunting beyond the tolerance set for the room.
- Motion: No camera whip, no dropped frames; IMU continuous where the rig has one.
- Sync: Every stream aligned within one frame; timestamps monotonic across cameras, state and action logs.
- Task compliance: Start and end states match the SOP; no skipped or reordered steps unless the protocol allows it.
- Duration: Inside the episode window for the task family; overruns are trimmed to the protocol, not the clip.
- Metadata: Environment, task, rig, operator id, session and consent reference present and valid on every episode.
- Privacy: No bystander faces, screens or documents without consent; blurred where the protocol flags it.
- Calibration: Intrinsics and extrinsics on file per rig and session for depth, stereo and multi-camera capture.
- Labels: Where annotation is ordered, agreement against the gold set above the threshold before the batch ships.