We turn ordinary videos of people handling food into motion a robot arm can learn. No code, no wearables, no teach pendant. Show the task once, and the robot takes it from there.
Teaching a robot a new food task usually takes an engineer reprogramming it, or an operator in a motion capture rig. We capture the task the way a worker already does it, and convert that motion into something any arm and gripper can learn.
A smartphone on a cap or a tripod films the task. We track both hands in 3D and separate the background, the food and the motion.
Finger opening becomes gripper opening. The hand path becomes joint motion. One recording drives 2, 3 or 5 finger grippers on 6 or 7 axis arms.
Augmented demonstrations and parallel simulation train the policy. On the line, weight feedback control checks every scoop against its target portion.
Robots learn manipulation from demonstrations, and demonstrations are expensive to collect with robots. Ours come from video, so every food task filmed adds training data that any robot arm can use.
Real videoOne camera, top down. The worker does the task at normal speed.
Real video21 keypoints per hand, every frame, with the objects tracked alongside.
The fingertip path is retargeted into a gripper path the robot can reach.
Inverse kinematics turns the path into joint commands, recorded as robot training data.
Hours of food handling data we plan to train on at each product stage. Targets, not yet collected.
A person moves cherry tomatoes on camera. Our pipeline extracts the hand motion and a simulated robot arm repeats it: approach, grasp, carry, release.
We remove the worker from the frame and move the objects around, so a single recording becomes many training conditions.
Motion is stored relative to the hand, so the data carries over when the robot or the gripper changes.
Two robot arm stations with RGB-D cameras are being assembled to collect real food handling data this fall.
Shape, size and position change with every item, so a robot cannot follow a fixed path the way it does with machined parts.
Seasoned vegetables clump, rice compresses, sauces drip. Physics engines struggle with this, so we learn it from real handling.
A lunch box line can switch dishes every morning. A new item has to be taught in minutes from a short demonstration.
Pick, place and scoop with a two finger gripper on existing conveyors.
Five finger hands and tactile data for finer, irregular handling.
Cooking, packing and transfer across arms and mobile manipulators.
Robots that prepare, cook and plate food on their own.
We are working with production sites, research partners and engineers who want to automate real food handling. Tell us what you make.
contact@dream-intelligence.ai