most robot datasets give the robot one instruction for a whole episode, like “clean the table”, and then a few minutes of motion.
the model has to figure out by itself where one step ends and the next begins which is hella hard.
I had the pleasure of chatting with jade choghari from scale AI, who walks us through FineART, a 1,700 hour two-arm robot dataset where every single step is labeled in plain words by a human (534k labels, no VLM in the loop!)
these subtask labels do a lot of heavy lifting to shape a proper policy.
without them the model can’t even reliably tell the left bin from the right one, even with a big vision-language backbone behind it!
with them, the steps it learned in words carry over when transfered to a completely different robot, to the point where it pulls off a long breakfast task it never practiced there and even reuses an “adjust” move it only ever saw on the previous robot!
very fun chat, we also get into how scale ai actually collects this stuff behind the scenes which is as impressive as the result themselves!
👉 Jade Website: https://jadechoghari.github.io/
👉 Jade GitHub: https://github.com/jadechoghari
👉 FineART (paper): https://arxiv.org/abs/2609.36416
👉 FineART-VLA (model): https://huggingface.co/lerobot/fineart_vla_base
Check out Arcee (American open research lab and sponsor of this interview):
♥️ https://www.arcee.ai/
Also trinity large thinking is available on open router:
♥️ https://openrouter.ai/arcee-ai/trinity-large-thinking#providers
♥️ and check out the NAC harness https://www.arcee.ai/blog/nac
Table of Content:
- 00:00:00 - what is FineART?
- 00:03:55 - my man jade choghari at ScaleAI SF HQ
- 00:06:34 - highschool days to machine learning research engineering
- 00:10:10 - the waterloo magic touch 🪄
- 00:10:50 - how did you get into lerobot?
- 00:12:46 - what drove jade into ai research?
- 00:15:01 - FineART: Data + VLA walkthrough
- 00:16:08 - “Divide each difficulty into as many parts as is feasible and necessary to resolve it” Rene Descartes
- 00:17:30 - how do you decide what a subtask should be?
- 00:21:56 - why full 100% human organic??
- 00:24:35 - dataset distribution
- 00:26:28 - how in the world did you pull off 100% human annotation in there man
- 00:31:05 - FineART-VLA policy + loss function
- 00:36:18: FineART-VLA interesting emerging behavior
- 00:42:00 - why does knowledge insulation protect the VLM?
- 00:45:15 - you think we could relax knowledge insulation once the expert is up to speed?
- 00:47:10 - what type of steps does the model get wrong?
- 00:51:31 - what context does the model get for each task?
- 00:55:20 - how are you handling error in the teleoperation setup?
- 01:01:00 - how many review passes does a label go through before it’s considered good?
- 01:04:00 - the surprising result in the breakfast task
- 01:07:40 - what makes skills transfer across robots just work?
- 01:09:50 - where did the model failed?
- 01:11:25 - does the model get lost on long tasks?
- 01:12:35 - why did ABC collapse while FineART kept climbing the benchmark?
- 01:14:17 - what’s next for FineART?
- 01:19:08 - how should you mix sim + teleop + human video data?
- 01:22:28 - is the VLM backbone here to stay?
- 1:26:25 - what is jade excited about these days?
Playback speed
×
Share post
Share post at current time
Share from 0:00
0:00
/
How to Make Better Robot Policies with Dense Subtask Labels?
we are interviewing Jade Choghari from Scale AI about how they labelled thousands of hours of data to make more robust robot policies!
learning with yacine show
a show about untangling the complexity of deep learning and the brain by interviewing smart researchers and builders in the field of intelligence!
ideal for scholars, industry practitioners or toddlers.
a show about untangling the complexity of deep learning and the brain by interviewing smart researchers and builders in the field of intelligence!
ideal for scholars, industry practitioners or toddlers.Listen on
Appears in episode
Recent Episodes









