Dreaming in Blocks — MineWorld, the Minecraft World Model

Microsoft has developed MineWorld, a real-time, open-source interactive world model for the game Minecraft. This model addresses the computational inefficiency of previous world models by using a parallel decoding algorithm to significantly increase the number of frames generated per second. The architecture combines a Vector Quantized Variational Autoencoder (VQ-VAE) for visual tokenization and a LLaMA-based transformer decoder to predict subsequent game states and actions. The paper also introduces a new metric for evaluating a world model's controllability, using an Inverse Dynamics Model to predict actions between frames. Quantitative results show that MineWorld outperforms its closed-source counterpart, Oasis, in both visual quality and interactive speed.