Skip to content

data_preparation.md

Latest commit

 

History

History
133 lines (91 loc) · 6.67 KB

data_preparation.md

File metadata and controls

133 lines (91 loc) · 6.67 KB
Raw

Preparing Datasets for GR00T-H

Prerequisites

Your data must be in LeRobot v2 format. If you're converting from v3, run scripts/lerobot_conversion/convert_v3_to_v2.py first. For the full format specification (parquet layout, episode structure, video conventions), see the Data Preparation Guide.

Each dataset needs a modality.json that maps raw parquet columns and index ranges to named keys (e.g., eef_pose, gripper). If your dataset stores delta commands or joint angles in the action column rather than absolute EEF poses, see the Copy-EEF pattern, which sources action data from observation.state at future timesteps.

Your embodiment must have a registered config. Built-in Open-H embodiments are auto-registered on import, so no extra setup is needed. For new embodiments, follow Adding Your Own Embodiment.

The Three Statistics Files

GR00T-H training relies on normalization statistics stored alongside each dataset. Three files live under meta/, each serving a distinct purpose.

meta/stats.json

Flat statistics computed directly from raw parquet data.

  • Contains: mean, std, min, max, q01, q99 per state and action key
  • Shape: (dim,) ... one value per dimension, no temporal component
  • Used for: state normalization (min-max or mean-std scaling)
  • Generated by: gr00t/data/stats.py

meta/temporal_stats.json

Per-timestep statistics computed after REL_XYZ_ROT6D action conversion. This is a GR00T-H addition for action normalization across multi-step horizons.

  • Contains: mean, std, q02, q98 per action key, per timestep in the action chunk
  • Shape: (horizon, dim) ... separate statistics for every step in the chunk
  • Used for: action normalization with normalization_type="temporal_meanstd"
  • Generated by: launch_finetune.py --calculate-norm-stats

Why temporal stats are beneficial

REL_XYZ_ROT6D computes cumulative displacements from the current EEF pose to each future timestep in the action chunk. The magnitude of those displacements grows with the prediction horizon:

  • At step 1 (33ms into the future at 30Hz), displacement is small.
  • At step 50 (1.7s into the future), cumulative displacement is much larger.

Flat normalization (a single mean and std across all steps) applies the same scaling to both, which can under-normalize early steps and over-normalize late ones. Temporal stats address this by providing each timestep with its own mean and std, calibrated to the actual displacement magnitude at that horizon. The released GR00T-H checkpoint was trained with these per-timestep statistics.

Here's what temporal_stats.json looks like in practice:

{
    "action": {
        "eef_pose": {
            "mean": [[0.001, 0.002, ...], [0.003, 0.005, ...], ...],
            "std":  [[0.01, 0.01, ...],  [0.02, 0.03, ...],  ...],
            "q02":  [[-0.02, -0.02, ...], [-0.05, -0.06, ...], ...],
            "q98":  [[0.02, 0.02, ...],  [0.05, 0.06, ...],  ...]
        }
    }
}

Each inner list has length equal to the action dimension (e.g., 9 for a single-arm EEF: 3 xyz + 6 rot6d). The outer list has horizon entries (e.g., 50 for a 50-step action chunk).

meta/relative_stats.json

Generated alongside stats.json. Contains per-timestep stats for RELATIVE (non-EEF) actions, with the same (horizon, dim) shape concept as temporal_stats.json but computed differently. You generally don't need to worry about this file. It's produced automatically during stats generation.

The prepare_datasets.sh Pipeline

Run the preparation script from the repository root:

bash open_h/prepare_datasets.sh \
    --embodiment-tag <EMBODIMENT_TAG> \
    --modality-json <path/to/modality.json> \
    /path/to/dataset_a /path/to/dataset_b ...

The script processes each dataset through three steps.

Step 1: Copy modality mapping

cp <modality.json> <dataset_path>/meta/modality.json

Places the column-to-key mapping where the data loader expects it. If the meta/ directory doesn't exist, the script creates it.

Step 2: Generate flat statistics

uv run python gr00t/data/stats.py \
    --dataset-path <dataset_path> \
    --embodiment-tag <EMBODIMENT_TAG>

Reads all parquet files, computes per-column statistics, and writes meta/stats.json (plus meta/relative_stats.json).

Step 3: Generate temporal statistics

uv run python gr00t/experiment/launch_finetune.py \
    --base-model-path nvidia/GR00T-H-N1.7 \
    --dataset-path <dataset_path> \
    --embodiment-tag <EMBODIMENT_TAG> \
    --calculate-norm-stats

This step loads the modality config for the embodiment (auto-registered, no --modality-config-path needed), iterates through episodes, performs the actual REL_XYZ_ROT6D conversion for each action chunk, computes per-timestep percentile statistics on the converted actions, and writes meta/temporal_stats.json.

Concrete Example: TUD TUNDRA UR5e

bash open_h/prepare_datasets.sh \
    --embodiment-tag TUD_TUNDRA_UR5E \
    --modality-json open_h/embodiments/tud_tundra_ur5e/modality_grasping_retraction.json \
    /path/to/Surgical/TUD/260131_TUNDRA_dataset/grasping_retraction

After running, verify the output:

ls /path/to/Surgical/TUD/260131_TUNDRA_dataset/grasping_retraction/meta/
# Expected: episodes.jsonl  info.json  modality.json  stats.json  tasks.jsonl  temporal_stats.json

Stats Merging at Training Time

When training on multiple datasets under the same embodiment tag (e.g., twelve dVRK datasets under jhu_imerse_dvrk), the training pipeline automatically merges their statistics. Each dataset keeps its own temporal_stats.json. At training time, gr00t/data/percentile_merge.py merges them, weighted by the mix ratios specified in the training config.

Troubleshooting

Problem Cause Fix
AssertionError: Embodiment tag already registered Passed --modality-config-path for a built-in tag Remove --modality-config-path; the config is auto-registered
IndexError: boolean index did not match indexed array along dimension 0 temporal_stats.json shape doesn't match current delta_indices Regenerate temporal stats with prepare_datasets.sh
FileNotFoundError: meta/temporal_stats.json Stats not generated before training Run prepare_datasets.sh before training
KeyError: '<tag>' not in MODALITY_CONFIGS Embodiment tag misspelled or new embodiment not registered Check spelling against EmbodimentTag enum; for new embodiments, ensure the config calls register_modality_config()