Rise: Toward Omni-Immersive Video Generation
From offline, to real-time, and now toward the OASIS — the fully immersive world envisioned in Ready Player One.
Inside Rise
Video Generation is Entering a New Phase
From offline generation to real-time generation, and now to omni-immersive generation, each paradigm represents a fundamental upgrade in how video models interact with users and the world.
Omni-immersive generation introduces a fundamentally different paradigm: the continuous video stream captured by the user's camera becomes a native input to the generative model itself.
The user's movements, expressions, and surrounding environment no longer need to be translated into explicit instructions; they directly participate in the generation of the world in real time. Users no longer need to learn how to formulate instructions that an AI can understand. The world begins to understand the user. This bidirectional interaction is a necessary step toward making video generation models genuinely intelligent.
The Limits of Two-Stage Video Generation
Today, most omni-immersive video generation systems rely on a two-stage pipeline consisting of a video understanding model followed by a video generation model. While this approach enables interactive video experiences, it introduces two fundamental limitations:
- Limited Generalization：In a two-stage pipeline, video understanding and video generation are modeled as separate systems. Their representations cannot be natively scaled together within a unified architecture, making it difficult to achieve end-to-end Native Representation Scaling across understanding and generation. As a result, adapting the system to different interaction scenarios often requires substantial engineering and task-specific tuning. This fundamentally limits the efficiency of the data feedback → model optimization → capability improvement loop.
- High Latency：The end-to-end latency of a typical two-stage interactive video system can reach 3–4 seconds, including the time required for the understanding model to interpret user input and produce rendering instructions, followed by the generation model rendering the corresponding frames. For the next generation of interactive content, this latency is simply too high. In the Gen Z era, users increasingly expect interactive experiences to respond within the range of hundreds of milliseconds, with around 500ms representing a critical threshold for maintaining a sense of real-time presence.
Two-stage video generation is therefore an intermediate form of omni-immersive technology — a representation of the upper bound of interactive experiences that the previous generation of architectures could reach.
A Unified Understanding-Generation Paradigm
We chose a harder but more fundamental path. Rather than connecting an understanding model and a generation model in sequence respectively, we build understanding and generation into a unified modeling paradigm within a single model.
This enables Native Interactive Representation Scaling, allowing user interaction signals to participate directly in the scaling of the video generative model. At the temporal level, every frame from the user's camera stream and every generated frame are jointly modeled within the same temporal context. Instead of first understanding the user and then generating a response, the model learns to understand and generate as part of the same continuous process. This unified architecture enables native generalization across diverse interaction scenarios while significantly reducing end-to-end latency.
An Omni-Immersive Foundation Model
We introduce Rise, the Omni-Immersive Foundation Model in the Era series.
Rise marks the beginning of a new era of immersive interaction. It represents a fundamental shift in video generation—from letting users experience a piece of content in real time to defining how users can enter a world with dramatically lower interaction costs.
Modeling a Living Racing World
Following large-scale pretraining on a diverse set of unified understanding-and-generation tasks, Rise was further post-trained on approximately 300 hours of high-speed racing video data, enabling it to learn realistic physical dynamics and simulate responsive physical feedback in high-speed racing environments. Powered by Native Interactive Representation Scaling, Rise enables users to control the racing world through a camera with an end-to-end latency of under 500ms.
But low latency is only the beginning. More importantly, Rise demonstrates emergent capabilities in both understanding and generation, without constraining how users enter the world.
Emergent Understanding Abilities
Users can interact with the racing world through steering wheels of arbitrary shapes, including everyday objects such as a mat or a smartphone.
They can also control the world simply through empty-handed gestures. The model generalizes across both first-person and third-person viewpoints.
Emergent Generation Abilities
Rise can generate a rich variety of racing vehicle appearances, allowing users to enter worlds with different visual identities. It also generalizes across fundamentally different environments, from closed racing circuits to open-world roads.
These results suggest that the model is not simply learning a fixed mapping between a specific input and a specific output. Instead, it is learning a more general representation of how users, environments, and generated worlds interact and evolve together.
From Experiencing Worlds to Entering Worlds
We believe the future of immersive content is not simply about rendering content in real time. It is about building worlds that can truly understand the people inside them.
Rise redefines how users enter a generated world: from learning instructions to expressing themselves through natural behavior; from operating a screen to participating in a world.
This is the next step for video generation—and the beginning of a new era of immersive interaction.
