Matrix-Game 4.0

OVERVIEW概览A harness around the video model用 Harness 包住视频模型

Interactive world models that only roll out pixels keep the world state implicit: once something leaves the view it is gone, and game rules have to be re-inferred from every frame. We wrap the video model in a harness that owns the state. The game engine evolves everything the rules can decide and renders it as a low-poly proxy. The Coding Agent takes over where rules run out: it turns a user's instruction into new rules registered into the engine, and retrieves the engine state to write prompts that carry the current state and the user's intent.只靠逐帧外推像素的交互式世界模型，世界状态都是隐式的：东西一离开视野就消失，游戏规则也要从每一帧里重新推断。我们给视频模型套上一层 Harness，由它持有世界状态。规则能决定的状态演化全部交给游戏引擎，引擎再把结果渲染成低模代理画面。规则管不到的地方由 Coding Agent 接手：它把用户指令写成新的规则、修改游戏引擎，并检索引擎状态，结合用户意图转述细化成提示词。

The video model then only has to render. It reads the per-frame proxy, a dynamic caption and a few reference frames, and generates one latent (four frames) per step with a causal DiT distilled to one or two denoising steps.于是视频模型只需要负责渲染。它读入逐帧的低模代理画面、一条动态描述和几张参考帧，用蒸馏到一到两步去噪的因果 DiT，每一步生成一个 latent（四帧）。

Game engine: rule-based state游戏引擎：按规则演化状态 Coding Agent: directing beyond rulesCoding Agent：新的规则与推理想象 Multi-reference appearance多参考帧锁定外观 Spatial memory by camera pose按相机位姿检索的空间记忆

METHOD方法How it works工作原理

Dynamic prompts动态 prompt

The harness writes the prompt from the engine state, one segment at a time. For this 40-second held-out pyramid run it reads the objects in view and the game events every 80 frames (12 segments): a scene summary with the player's state, the enemies and landmarks around, and the events of that segment. The prompt panel shows what the harness writes for the segment being played.Harness 按引擎状态逐段写 prompt。这段 40 秒的金字塔留出例每 80 帧读一次画面里的物体和游戏事件，共 12 段，每段包括场景总述（玩家状态、周围的敌人和建筑）和这一段里发生的事件。prompt 栏显示 Harness 为正在播放的那一段写出的 prompt；原文是英文，这里是中文对照，切到 English 可看原文。

Prompt segment当前 prompt 段1 / 12

player action玩家动作 enemy action敌人动作 hit / explosion命中与爆炸 enemy defeated击败敌人 enters / exits view进入 / 离开画面 player state玩家状态

Prompt · written by the harness

Scene summary

Events in this segment

Prompt · 中文对照

场景总述

本段事件

Rolling KV cache, one chunk at a time滚动 KV 缓存，一次生成一个块

Each new chunk attends to retrieved spatial memory, the reference frames and a sliding window of recent chunks. The oldest chunk leaves the window as a new one arrives.每个新块都关注检索到的空间记忆、参考帧，以及最近若干块组成的滑动窗口。新块进来时，最旧的块离开窗口。

spatial memory空间记忆 reference frames参考帧 recent chunks最近的块 chunk being generated正在生成的块

RESULTS结果From proxy to picture从低模到画面

Four games, one clip each四个游戏，各一例

One held-out clip per game, 15 seconds each. Left of the divider is the generated frame, right of it is the engine's low-poly proxy. Drag the divider with a mouse or a finger. Both halves are cut from the same video frame, so they never drift apart.每个游戏取一段留出评测片段，每段 15 秒。分割线左侧是生成画面，右侧是引擎低模，用鼠标或手指拖动分割线即可对比。两半取自同一帧视频，始终逐帧同步。

LONG PLAY长时间游玩One uninterrupted session per game每个游戏一段不间断的游玩

Three continuous sessions, 54 to 99 seconds each. Left of the divider is the generated frame, shown with the game HUD and the key inputs; right of it is the engine's low-poly proxy. Drag the divider to compare, and turn the sound on to hear the game audio.三个游戏各一段不间断的游玩，每段 54 到 99 秒。分割线左侧是生成画面，带游戏界面和当时的按键；右侧是引擎低模。拖动分割线对比，点「打开声音」可以听游戏音效。

Code builds the game, AI renders the world in real time代码搭建游戏，AI 实时渲染世界