Inference-time Physics Alignment
of Video Generative Models with Latent World Models

Jianhao Yuan1,2, Xiaofeng Zhang1,3,4,‡, Felix Friedrich1,‡, Nicolas Beltran-Velez1,5,‡, Melissa Hall1
Reyhane Askari-Hemmat1, Xiaochuang Han1, Nicolas Ballas1, Michal Drozdzal1,†, Adriana Romero-Soriano1,3,6,7,†
1 FAIR, Meta Superintelligence Labs 2 University of Oxford 3 Mila — Québec AI Institute 4 Université de Montréal
5 Columbia University 6 McGill University 7 Canada CIFAR AI Chair
‡ Equal contribution † Joint last author
CVPR 2026 · Highlight
Paper arXiv Supplementary Code Video Poster
TL;DR. A pretrained latent world model (VJEPA-2) is a strong physics-plausibility reward. We use it at inference time — no fine-tuning of the generator — to steer video diffusion toward physically plausible samples, achieving state-of-the-art on the PhysicsIQ benchmark across MAGI-1, Sora 2, and a video latent diffusion model.
Figure 1 — qualitative samples across four physics domains

Figure 1. Left: WMReward achieves new state-of-the-art on the PhysicsIQ benchmark, improving physics plausibility on top of strong open-source (vLDM, +5.68) and frontier (Sora 2, +4.13; MAGI-1, +6.78) backbones — a final score of 62.64%, +7.42 over Sora 2. Right: Performance scales gracefully with the number of particles used in our reward-guided sampling, while pixel-reconstruction (VideoMAE) and VLM-logit (Qwen3-VL) reward signals plateau or degrade.

Abstract

State-of-the-art video generative models produce promising visual content yet often violate basic physics principles, limiting their utility. While some attribute this deficiency to insufficient physics understanding from pre-training, we find that the shortfall in physics plausibility also stems from suboptimal inference strategies. We therefore introduce WMReward and treat improving physics plausibility of video generation as an inference-time alignment problem. In particular, we leverage the strong physics prior of a latent world model (here, VJEPA-2) as a reward to search and steer multiple candidate denoising trajectories, enabling scaling test-time compute for better generation performance.

Empirically, our approach substantially improves physics plausibility across image-conditioned, multiframe-conditioned, and text-conditioned generation settings, with validation from human preference study. Notably, in the ICCV 2025 Perception Test PhysicsIQ Challenge, we achieve a final score of 62.64%, winning first place and outperforming the previous state of the art by 7.42%. Our work demonstrates the viability of using latent world models to improve physics plausibility of video generation, beyond this specific instantiation or parameterization.

Qualitative Samples

In each panel, rows are top: ground truth · middle: baseline · bottom: WMReward. Examples cover rigid-body dynamics, fluid mechanics, object permanence, and buoyancy.

Qualitative comparison — set 1
Set 1 — rigid body & object interaction.
Qualitative comparison — set 2
Set 2 — fluids & buoyancy.

Method

WMReward — surprise as a physics-plausibility reward

A latent world model encodes video into compact representations and learns a transition function over them, yielding a representation that focuses on physically consistent dynamics rather than pixel statistics. We use the cosine distance between predicted and observed future latents as our reward:

r(x) = ⟨ 1 − cos( ẑfuture, zfuture ) ⟩

WMReward — VJEPA-2 surprise reward

Three inference-time sampling schemes

We draw from a reward-tilted distribution with three complementary schemes — Best-of-N samples N candidates and keeps the highest-reward video; Guidance (∇) uses reward gradients during denoising via Tweedie's formula; and WMReward (∇+BoN) combines both (default).

Headline Results

WMReward delivers consistent gains across model families on the PhysicsIQ benchmark.

+5.68
on vLDM
+4.13
on Sora 2
+6.78
on MAGI-1
62.64%
final score — new SoTA on PhysicsIQ
(+7.42 over Sora 2)

See 5-minute walkthrough for a guided tour of the method and results.

5-Minute Presentation Video

BibTeX

@article{yuan2026inference,
  title   = {Inference-time Physics Alignment of Video Generative Models with Latent World Models},
  author  = {Yuan, Jianhao and Zhang, Xiaofeng and Friedrich, Felix and
             Beltran-Velez, Nicolas and Hall, Melissa and Askari-Hemmat, Reyhane and
             Han, Xiaochuang and Ballas, Nicolas and Drozdzal, Michal and
             Romero-Soriano, Adriana},
  journal = {arXiv preprint arXiv:2601.10553},
  year    = {2026}
}

Acknowledgements

We thank our collaborators and reviewers for valuable feedback. This project page template is adapted from the Academic Project Page Template by Eliahu Horwitz.