- likes
- 104
- comments
- 5
Post
How does labeled data become a model back on the road? Shiyan Xu, an engineer on our Data Engine team, breaks down how we take labeled data through curation, scenarios, data loading, training, and evaluation, all the way back to the fleet, across hundreds of petabytes of multimodal data from automotive, trucking, mining, and robotics fleets. Closing that loop comes down to a few key design choices: 👉 Choosing segments to feed the model: Curation runs selection criteria, like nighttime driving with a cut-in under two seconds, against a curation table hundreds of columns wide. Scenarios no metric or label covers can be found by searching clip embeddings with text or an example clip. 👉 Building scenarios for training: Each scenario bundles aligned frames, labels, map, and route into one row. Lance’s blob v2 API makes each scenario’s blob payload easy to work with: written and versioned with the row, read only when needed. Data evolution adds new fields without rewriting existing files. 👉 Turning scenarios into model inputs: Lance’s random access lets training sample rows non-sequentially in small, scattered batches, with each batch fetched as a single take. Checkpointing lets a restarted job pick up where the sampler left off instead of replaying. 👉 Training in two stages: Imitation learning first teaches the model to drive like the fleet’s drivers, then reinforcement...
Keep reading with a free account
The rest of this post, and every signal for Applied Intuition, is in your free account.