DGX Spark / GB10 fork — inference port and stability fixes for
sm_121/ aarch64. See FORK.md for what's different from upstream and the verified operating envelope. See BENCHMARKS.md for per-step FPS measurements and rendered test clips. Upstream: https://github.com/IamCreateAI/Dreamerv4-MC⚠️ Operating note:steps_size≥ 16 produces rapid scene decoherence on this checkpoint; live and cold-offline use should stay onsteps_size in {4, 8}. See BENCHMARKS.md for measurements.
GitHub README markdown strips raw HTML5 <video> tags, so the directly viewable embedded gallery lives in a checked-in HTML page served through GitHub Pages:
- Open the embedded video gallery — served from the
gh-pagesbranch. - RawGitHack mirror — fallback if Pages is unavailable.
- Gallery source in this repo.
See BENCHMARKS.md for settings, measurements, checksums, and the full clip list.
If you are reviewing or running this GB10 fork, start here. This fork targets NVIDIA DGX Spark / GB10 (sm_121, aarch64) inside an NGC PyTorch container. The upstream Installation section further down clones IamCreateAI/Dreamerv4-MC and builds flash-attn from source on x86_64 + CUDA 12.1 — do not use that path for GB10.
Prerequisites
- NVIDIA DGX Spark / GB10 (or another
sm_121aarch64 host) - Docker with the NVIDIA Container Toolkit (
--gpus allworks) - ~12 GB available accelerator / unified memory (1.7 B dynamic model + 430 M tokenizer)
- Internet access on first run (pulls the NGC image and pip wheels into the container)
Setup
# 1. Clone THIS fork (not upstream)
git clone https://github.com/blackfirebitcoin/Dreamerv4-MC-GB10.git
cd Dreamerv4-MC-GB10
# 2. Download checkpoints from upstream HuggingFace into ./checkpoints/
# Required layout: checkpoints/dynamic/ checkpoints/tokenizer/
# https://huggingface.co/IamCreateAI/Dreamerv4-MC
# 3. Start the live inference container at the recommended steps_size=4
scripts/ngc/start-live.sh 4
# 4. Open the browser UI (or tunnel from your dev machine if remote)
# http://localhost:8765 — Press Z to lock the mouse, WASD to move, V to refresh the scene.Boot takes ~30 seconds (pip install inside the container + model load + CUDA-graph capture). Tail per-frame timing with:
docker logs -f dreamerv4-ngc 2>&1 | grep -E 'TIMING|ACTION'Other run modes (offline render, capture/replay, high-step warmup-then-record): see scripts/ngc/ and the Quickstart section in FORK.md. For the verified per-step operating envelope and rendered demo MP4s, see BENCHMARKS.md.
The original upstream README continues below.
This repository contains the Inference Code for the Minecraft Autoregressive World Model. This project serves as an open-source reproduction of the DreamerV4 architecture, tailored specifically for high-fidelity simulation in the Minecraft environment. Our model utilizes a MAE (Masked Autoencoder) for efficient video compression and a DiT (Diffusion Transformer) architecture to autoregressively predict future game frames based on history and action inputs in the latent space. This codebase is streamlined for deployment and generation, supporting long-context inference and real-time interaction.
- Inference Only: Lightweight codebase focused on generation, stripped of complex training logic.
- Long Context Support: Capable of loading Long-Context models to recall events from 12 seconds prior.
- Fast Inference Backend: Built-in optimized inference pipeline designed for high-performance, real-time next-frame prediction.
- Infinite Generation: Supports infinite generation without image quality degradation during long-term rollouts.
- Complex Interaction: Supports a variety of interactions within the Minecraft world, such as eating food, collecting water, using weapons, etc.
Please download the pre-trained weights and place them in the checkpoints/ directory before running the code.
| Model Name | Params | VRAM Req | Description |
|---|---|---|---|
| MAE-Tokenizer | 430M | >2GB | Handles video encoding and decoding. |
| Dynamic Model | 1.7B | 9GB | Generates the next frame based on history and action. |
🔗 Download: HuggingFace Collection
We recommend using Python 3.10+ and CUDA 12.1+.
# 1. Clone the repository
git clone https://github.com/IamCreateAI/Dreamerv4-MC.git
cd Dreamerv4-MC
# 2. Create a virtual environment
conda create -n dreamer python=3.12 -y
conda activate dreamer
# 3. Install PyTorch (Adjust index-url for your CUDA version)
pip install torch torchvision --index-url [https://download.pytorch.org/whl/cu121](https://download.pytorch.org/whl/cu121)
# 4. Install dependencies
pip install -r requirements.txt
MAX_JOBS=4 pip install flash-attn --no-build-isolation
pip install -e .python ui/inference_ui.py --dynamic_path=/path/to/dynamic_model \
--tokenizer_path=/path/to/tokenizer/ \
--record_video_output_path=output/Coming Soon
| Key | Action |
|---|---|
| W / A / S / D | Move |
| Space | Jump |
| Left Click | Attack / Destroy |
| Right Click | Place / Use Item |
| E | Open/Close Inventory (Simulation) |
| 1 - 9 | Select Hotbar Slot |
| R | start/stop record the video |
| V | refresh into new scene |
| left Shift | Sneak |
| left ctrl | Sprint |
If you use this codebase in your research, please consider citing us as:
@article{hafner2025dreamerv4,
title = {Dreamer-MC: A Real-Time Autoregressive World Model for Infinite Video Generation},
author = {Ming Gao, Yan Yan, ShengQu Xi, Yu Duan, ShengQian Li, Feng Wang},
year = {2026},
url = {https://findlamp.github.io/dreamer-mc.github.io/}
}as well as the original Dreamer 4 paper:
@misc{Hafner2025TrainingAgents,
title={Training Agents Inside of Scalable World Models},
author={Danijar Hafner and Wilson Yan and Timothy Lillicrap},
year={2025},
eprint={2509.24527},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2509.24527},
}This project is built upon the following foundational works:
- MaeTok: Masked Autoencoders Are Effective Tokenizers for Diffusion Models (Chen et al., ICML 2025)
- DreamerV4: Training Agents Inside of Scalable World Models (Hafner et al., 2025)
- CausVid: From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (Yin et al., CVPR 2025)