Video-Action Models for Data-Efficient Industrial Manipulation
Eugenio Chisari (1,*), Stefanos Charalambous (1,*), Oliver Sanchez (1,*), Thomas Umbach (1), Michael Gentner (2), Fabio Quattrini (1), Norica Bacuieti (1), Sabin Grube Doiz (2), Valentin Bertle (2), Sigmund Hennum Hoeg (1), Alain Ryser (1), German Rodriguez (1), Mario Arduini (1), Dang Nguyen (1), Florian Aumann (1), Felix Hegg (1), Manuel Meier (1), Philipp Wand (1), Julian Lotzer (1), Alessio Desogus (1), Benedek Forrai (1), Michael Bloesch (1), Stephan-Daniel Gravert (1), Stefan Weirich (1), Patrick Lanners (2), Elvis Nava (1) / September 23, 2026
(1) Mimic Robotics · (2) BMW Group · (*) Equal contribution
Learning industrial manipulation from pretrained video representations and limited action-labeled demonstrations.
Industrial manipulation tasks from our automotive data collection, shown across the three collection sources used for pretraining: teleoperation of the dexterous bimanual platform (top), wearable data collection with human operators (middle), and teleoperation of a Trossen setup (bottom).
FLUX-mimic combines a pretrained video backbone with an action expert, trained jointly to predict future video and robot actions.
Industrial manipulation with less action data
Industrial manipulation requires precise alignment and controlled handling of rigid, deformable, and flexible components. Access to specialized hardware, parts, and fixtures makes demonstration collection and evaluation difficult.
For this project, we collected an industrial manipulation dataset covering 31 tasks and 295.8 hours of recordings, using production components from automotive manufacturing. Robot teleoperation provides action-labeled demonstrations, while camera-only wearable recordings contribute additional video for backbone pretraining.
We introduce FLUX-mimic, built on a backbone pretrained on large-scale web and industrial task data. An action expert uses the backbone’s features to generate robot trajectories, with joint video-action supervision during policy training.
We evaluate separately trained policies on two bimanual platforms: Franka arms with dexterous hands and Trossen arms with grippers.
Industrial Dataset
We collected 55,217 episodes across 31 tasks, totaling 295.8 hours of industrial manipulation recordings.
Recorded time per task, grouped into four benchmark tasks (left) and 27 additional pretraining tasks (right), stacked by collection source: wearable, Franka with dexterous hands, and Trossen teleoperation. All tasks are used for video-only backbone pretraining; benchmark-task recordings are also used for policy training and evaluation.
- Robot teleoperation
- Using the Franka and Trossen platforms, we collected 40,098 teleoperated demonstrations across 20 tasks, totaling 218.9 hours. Each demonstration includes camera observations and the corresponding robot actions.
- Camera-only wearable recordings
- 15,119 episodes and 76.8 hours, including 11 tasks absent from the teleoperation collection. These recordings have no action labels and are used for video pretraining only.
From video representations to robot actions
FLUX-mimic architecture. We condition the video backbone with the task caption and the past video frames and the action expert with the past robot state. Then, we run one video denoising step and layer-wise attend to all video features with our action expert to predict robot actions. We train with joint video-action loss.
- Video pretraining
- FLUX 3 provides representations learned from images, video, and audio, including web and industrial manipulation data. Camera-only wearable recordings contribute to video pretraining.
- Joint policy training
- Video and action predictions use flow matching objectives. Their losses receive equal weight, and both the pretrained backbone and action expert are updated.
- Efficient action generation
- A single backbone denoising step supplies features for the action expert. Predicting actions does not require sampling a complete video.
Evaluation Results
Production components bring tight clearances, deformable materials, and constrained routing. Policies are trained separately for each embodiment.
Control unit assembly (left) and control unit fixture insertion (right)
Soft part kitting (left) and wire harness routing (right)
- Control unit assembly
- The robot retrieves a control unit from a box and mounts it onto an aluminium bracket. This task requires grasping the unit and accurately aligning it with the bracket during mounting.
- Control unit fixture insertion
- The robot inserts a control unit component into a rigid fixture. The task requires aligning the component with the fixture and maintaining this alignment during insertion.
- Soft part kitting
- The robot picks deformable foam strut covers from a bin and places them into ordered slots. The task combines grasping from an unstructured source arrangement with placement of soft parts. We also use this task for the sample-efficiency experiment and the video loss ablation.
- Wire harness routing
- The robot routes a flexible cable along a door panel and fixes it at the required locations. Successful execution requires handling the cable while positioning it relative to the panel geometry.
High success with limited action data
Sample efficiency on soft part kitting with Franka. Success rate (final stage) as a function of action-data budget, from 1 h to 32 h. Each checkpoint is evaluated over 40 rollouts. Markers show the empirical success rate; shaded lobes show the posterior distribution over the true success rate.
In our soft part kitting experiments on Franka, FLUX-mimic achieved a success rate of 77.5% when trained with two hours of action-labeled demonstrations. Increasing the training data to eight hours brought success to 95%.
FLUX-mimic exceeds both baselines’ full-data performance using one quarter of the action-labeled demonstrations. Performance plateaus at larger budgets, while the baselines require more action data to approach it.
These budgets cover policy training and exclude backbone pretraining, as in the paper.
Video prediction loss during training
We evaluate whether adding a video prediction loss during policy training improves performance on soft part kitting with Franka.
Effect of the video prediction loss. Full-task success rate on soft part kitting with and without the video prediction loss, each evaluated over 20 rollouts. Markers show the empirical success rate; shaded lobes show the posterior distribution over the true success rate.
Adding the video prediction loss during training improves soft part kitting success by 35 percentage points.
Both variants use multitask policies trained on the full dataset, start from the same pretrained video backbone, and update the backbone during training. Each is evaluated over 20 rollouts.
At inference, both variants predict actions from the backbone’s features without generating a video.
Acknowledgments
We thank Nebius for providing part of the compute used in our experiments.