AI Data

Delivery Formats for Robotics Datasets: MP4, JSON Metadata, LeRobot and RLDS

By Chisom Anen18 November 20266 min read
Delivery Formats for Robotics Datasets: MP4, JSON Metadata, LeRobot and RLDS

Overview & Key Takeaways

AI and robotics teams need data in specific formats to train models effectively. This article explains common delivery formats, including MP4 video, JSON metadata, and specialized formats like LeRobot and RLDS, detailing what they are and why they matter for building intelligent systems.

Key Takeaways
  • AI training data for robotics often includes video, structured metadata, and specialized file formats.
  • MP4 files capture the visual information, while JSON metadata provides context for each video clip.
  • LeRobot and RLDS are standardized formats designed to simplify the ingestion and processing of robotics data.
  • Choosing the right data format ensures compatibility with AI training frameworks and accelerates model development.
  • TrustAm Data provides data in flexible formats, including MP4, JSON, and can adapt to LeRobot and RLDS upon request, ensuring AI teams receive data ready for immediate use.

Why Data Formats Matter for Robotics AI

Building sophisticated AI and robotics systems requires more than just raw information; it demands data delivered in formats that machines can easily understand and process. For robotics, this typically means combining visual data with structured contextual information. The way this data is packaged, delivered, and organized significantly impacts how efficiently AI teams can train models for tasks ranging from object recognition to complex manipulation. Understanding these formats is crucial for any team looking to source high-quality, usable data.

When AI teams commission data, they need it to be directly compatible with their existing tools and workflows. This means specifying not just the content of the data, but also how it should be delivered. Different robotics applications may have different needs. For instance, a robot learning to navigate a busy Lagos market requires video of the environment and precise location data, while a robot learning to assemble a product needs detailed step-by-step instructions alongside the visual actions. Asking the right questions about data formats upfront with a vendor is key to avoiding costly integration issues later.

Nigerian engineer examining a robotic arm
Data formats are essential for training robots to perform tasks like those involving robotic arms.

Standard Delivery: MP4 Video and JSON Metadata

The most common delivery format for visual AI training data is MP4 video files, often paired with JSON metadata. MP4 is a widely supported container format that efficiently stores video and audio streams. For robotics, these MP4s are typically first-person (egocentric) recordings of tasks being performed, capturing the agent's perspective. This type of data is valuable for training models that need to understand actions and environments from a human or robot's point of view, as seen in projects like Ego4D: Around the World in 3,000 Hours of Egocentric Video [arXiv].

Accompanying the video are JSON (JavaScript Object Notation) files. JSON is a lightweight data-interchange format that is easy for humans to read and write, and easy for machines to parse and generate. In the context of AI training data, each JSON file typically corresponds to an MP4 video clip and contains crucial metadata. This metadata can include:

  • Timestamps for specific actions within the video.
  • Labels for objects, actions, or events.
  • Pose information for human or robot bodies.
  • Sensor readings (e.g., IMU, GPS) synchronized with the video.
  • Information about the environment, participants, and task.
  • Consent IDs for speakers or participants, as per data privacy regulations like the Nigeria Data Protection Act (NDPA) [NDP Act GAID 2025].

This structured metadata is vital. Without it, raw video is difficult for AI models to learn from effectively. For example, knowing exactly when a specific tool is picked up or when a particular instruction is given within a video clip is essential for supervised learning. TrustAm Data provides all approved recordings as 1080p MP4 files, each with a corresponding JSON metadata file detailing the content, ensuring that AI teams have the context they need.

Specialized Formats: LeRobot and RLDS

While MP4 and JSON are standard, the AI and robotics community has developed more specialized formats to streamline data handling further. Two prominent examples are LeRobot and RLDS (Robo-school Language Data Specification).

LeRobot, hosted on Hugging Face [GitHub], is a format designed to standardize the representation of robotics data. It aims to make it easier for researchers and developers to share, discover, and use robotics datasets. LeRobot often involves structured arrays and dictionaries that clearly define actions, observations, and other relevant data points, making it highly compatible with popular machine learning libraries.

RLDS (Robo-school Language Data Specification), developed by Google Research [GitHub], is another specification for representing robotics data. RLDS focuses on creating a standardized schema that can capture a wide range of robotic interactions, including sensor data, actions, and environment states. Its goal is to facilitate interoperability between different datasets and robotics platforms, simplifying the process of building and deploying AI models for robots.

These formats are particularly useful for large-scale projects and multi-modal datasets. For instance, the Open X-Embodiment project [arXiv], which aims to create a massive dataset for robotic learning, uses standardized formats to enable diverse robots to contribute and learn from the shared data. Similarly, models like π0, a vision-language-action flow model for general robot control, benefit from structured data representations [arXiv].

Nigerian hands typing code on a laptop
Structured data formats like LeRobot and RLDS help AI teams process and train models efficiently.

TrustAm Data: Flexible Delivery for AI Teams

At TrustAm Data, we understand that AI and robotics teams have diverse needs when it comes to data formats. Our primary delivery is in MP4 video and corresponding JSON metadata, offering a robust and widely compatible solution. This format is ideal for many applications, including training models for tasks that involve human activities, as captured in datasets like EPIC-KITCHENS [EPIC-KITCHENS].

We also recognize the growing importance of standardized formats like LeRobot and RLDS. While these are not our default delivery, we can accommodate requests to convert our collected and processed data into these formats. This flexibility ensures that whether you are working with standard ML frameworks or require data structured for specific robotics platforms, TrustAm Data can supply what you need. Our goal is to provide data that requires minimal integration effort, allowing your team to focus on model development and innovation. This aligns with the broader push for standardized and ethically sourced data in AI, as highlighted by initiatives like the EU AI Act's emphasis on data governance [EU AI Act].

We offer a free 10-hour sample of data collected to your specific requirements. This allows you to evaluate the quality, relevance, and importantly, the suitability of our delivery formats for your project before committing to a larger purchase. We can also discuss custom data collection projects tailored to unique specifications, ensuring you get the precise data needed for your AI and robotics applications.

Frequently Asked Questions

What is the most common format for AI training data?
The most common formats for AI training data, especially for visual tasks, are MP4 video files paired with JSON metadata. MP4 stores the visual information, while JSON provides structured context such as labels, timestamps, and sensor data, making the data usable for machine learning models.
Why are specialized formats like LeRobot and RLDS important?
LeRobot and RLDS are important because they standardize the representation of robotics data, simplifying data sharing, discovery, and integration into AI training pipelines. They aim to ensure compatibility across different datasets and robotics platforms, accelerating the development of AI models for robots.
Can TrustAm Data provide data in LeRobot or RLDS formats?
Yes, while our standard delivery is MP4 video with JSON metadata, TrustAm Data can convert our collected and processed data into LeRobot or RLDS formats upon request. We strive to be flexible to meet the specific technical requirements of AI and robotics teams.
What kind of metadata is included with TrustAm video data?
Our JSON metadata typically includes timestamps for key events, labels for objects and actions, pose information, synchronized sensor readings, details about the environment and task, and consent IDs, ensuring comprehensive context for each recorded minute.
How can I test the data formats before purchasing?
TrustAm Data offers a free 10-hour sample of data collected to your specifications. This sample allows you to thoroughly evaluate the video quality, metadata accuracy, and the suitability of our delivery formats for your AI and robotics projects.
Buy from TrustAm Data

Need AI training data from Nigeria?

Egocentric video of skilled work and Nigerian-language speech, collected to order. Email [email protected] or request a free 10-hour sample; a person replies within one working day.

CChisom Anen

Founder and CEO of TrustAm. Building Nigeria's reputation marketplace for verified service providers. Reviews unlock only after real payments.

Disclosure: This article is published by TrustAm Data, part of TrustAm Limited. Links to /data pages are our own data service.

Sources & References

  1. [arXiv]— arxiv.org
  2. [NDP Act GAID 2025]— ndpc.gov.ng
  3. [GitHub]— github.com
  4. [EPIC-KITCHENS]— epic-kitchens.github.io
  5. [EU AI Act]— artificialintelligenceact.eu

Related Articles