The Data Flywheel

The most reliable way to improve a robot learning policy is not to change the model — it is to improve the data. The data flywheel is the core iteration loop for any serious robot learning project:

1

Evaluate

Run 20 trials. Measure success rate. Categorize failures by type. You did this in Unit 5.

2

Identify the primary failure mode

Is it data quality (inconsistent demos), distribution shift (unseen positions), or model capacity (precise enough trajectory but wrong)? Your Unit 5 diagnosis answers this.

3

Collect targeted data

Record 20–30 demonstrations specifically covering the failure regime. If the policy fails at objects on the left side of the workspace, record 20 demos of that specific position. Do not record more of what is already working.

4

Retrain and re-evaluate

Merge the new data with your existing dataset, retrain, and run the 20-trial evaluation again. Expect a 10–20 percentage point improvement per cycle when the diagnosis was correct.

Mixing Datasets

LeRobot can train on multiple datasets simultaneously, which is useful for combining your targeted collection data with your original dataset — or even with public community datasets for the same task and robot type.

# Merge two datasets into a new combined dataset python -m lerobot.scripts.push_dataset_to_hub \ --dataset-dir ~/lerobot-datasets/pick-place-v1 \ --repo-id $HF_USER/pick-place-v2-merged # OR train directly on multiple repo IDs python -m lerobot.scripts.train \ --policy-type act \ --dataset-repo-id "$HF_USER/pick-place-v1,$HF_USER/pick-place-targeted" \ --dataset-repo-id-weights "1.0,2.0" \ --output-dir ~/lerobot-policies/pick-place-v2 # The weights parameter upsamples the targeted data 2x # relative to the original dataset
Mixing with public datasets: Before mixing in a community dataset, verify that the robot type and action space dimensions match yours. Mixing a 7-DOF dataset into a 6-DOF training run will cause a silent shape mismatch error. Always inspect the info.json of any dataset you plan to mix.

Share Your Model on HuggingFace Hub

Sharing your trained model makes it available to the community and lets others use your policy as a starting point. Models shared in the standard LeRobot format can be loaded directly by anyone with pip install lerobot.

# Push your best checkpoint to HuggingFace Hub python -m lerobot.scripts.push_policy_to_hub \ --checkpoint-path \ ~/lerobot-policies/pick-place-v1/checkpoints/step_050000 \ --repo-id $HF_USER/act-pick-place-so100 # Add a model card (recommended) # The push command creates a README.md template — fill it in with: # - Robot type and task description # - Training dataset repo ID # - Evaluation success rate # - Video of the policy running on your robot

Share Your Dataset with the Community

Your dataset (which you pushed in Unit 3) is already on HuggingFace Hub. To make it more discoverable and useful to others:

  • Add a dataset card on HuggingFace — describe the task, robot, recording setup, and number of episodes. This is the single most impactful thing you can do for dataset discoverability.
  • Tag it with lerobot, your robot type (e.g., so100), and your task category (e.g., pick-and-place).
  • Submit it to the SVRC dataset library for curation and inclusion in the community index.

Contributing Hardware Configs Back to LeRobot

If you added a custom hardware config for an unsupported robot in Unit 1, consider contributing it back to the LeRobot repository. Open a pull request to huggingface/lerobot with your config file in lerobot/configs/robot/. The maintainers review hardware contributions quickly and this directly benefits every future user of that hardware.

What's Next: More Capable Policies

You now have the complete LeRobot workflow. Here is where to go from here:

Language

SmolVLA — language-conditioned policies

Switch from ACT to SmolVLA when you want the policy to respond to natural language instructions ("pick up the blue block") or generalize across tasks. Requires ~200+ demonstrations and a more structured prompt format. See the SmolVLA paper and LeRobot examples.

Scale

Pi0Fast — high-speed VLA inference

Pi0Fast runs at 100Hz inference (vs ACT's 30Hz), enabling faster manipulation tasks and tighter control loops. It requires a GPU at inference time but produces significantly more dexterous behavior at scale.

Generalization

Multi-task policies

Train a single policy on multiple tasks using a mixed dataset with different task_index values per episode. SmolVLA and Pi0Fast both support multi-task training natively. See the LeRobot multi-task recipe in the GitHub examples.

You completed the LeRobot path.

You installed LeRobot, understood the dataset format, recorded your own demonstrations, trained an ACT policy, evaluated it systematically, and ran the data flywheel. That is the complete open-source robot learning workflow — the same one used by research labs and robotics startups worldwide.

Questions? Join the HuggingFace Discord #lerobot — the maintainers and community are active and welcoming.