Method
Two complementary noise-warping mechanisms compose a single 3D-grounded motion-consistent latent. Click any card to expand.
Local Warping via 3D Point Tracks
Project sparse 3D tracks to the latent grid and propagate noise by copying a 3×3 patch (R = 2) along each trajectory. Collisions resolved with a depth-aware z-buffer — the closer track wins, enforcing plausible occlusion.
Sparse tracks carry explicit depth, robust to occlusion and viewpoint change. The result is a partial, motion-visible noise prior that already encodes object dynamics before any denoising step is taken.
View-Consistent Sphere Sampling
A virtual 3D sphere centered at the first camera, coated with a latitude–longitude Gaussian noise texture. For each frame, cast rays from the camera and sample noise at the sphere intersections — view-consistent noise for every direction.
Fills the regions track-the-noise cannot cover (disocclusion under camera motion). Sphere radius = max(α·trajectory-spread, scene-depth) with α = 1.2, guaranteeing the camera path stays inside.
Combine Local and Global Noise
Initialize the latent with Track-the-Noise; fill uncovered pixels with Move-the-World. A small Gaussian-noise blend (β = 0.2) restores temporal Gaussianity without hurting motion fidelity.
The constructed noise is spatially Gaussian by construction, but not temporally Gaussian; the β = 0.2 blend is the sweet spot in the paper's ablation — quality stays best while EPE drops.
LoRA on a Frozen 14B Backbone
Train LoRA rank 64 on Wan 2.1 I2V (14B) with flow matching, 10K steps. Preprocessing (ViPE depth/pose + TAPIP3D 3D tracks) is offline; at inference, the user draws 3D tracks in a 3D GUI and we warp the noise — ~0.2 s overhead, no extra parameters.
The same noise pipeline supports independent and joint control: motion-only, camera-only, or both. No control adapters, no flow networks, no inference-time modules added.