6arXiv cs.AI (Artificial Intelligence)·
World from Motion: generative dynamic 3D Gaussian reconstruction from monocular video
Researchers present World from Motion, a method that generates freely renderable dynamic 3D Gaussian Splatting (3DGS) representations from monocular video by conditioning a video model on pixel-aligned renderings encoding appearance, geometry, and 3D scene motion. The approach trains on aligned multiview video pairs with simulated monocular reconstruction artifacts, then distills model generations back into a consistent dynamic 3DGS at test time. The method claims state-of-the-art results in 4D reconstruction and generalizes to in-the-wild videos with large viewpoint changes and dynamic content.
Related guides (1)
Related events (8)
Introduction to 3D Gaussian Splatting
A Hugging Face blog post introduces 3D Gaussian Splatting, a technique for real-time novel view synthesis and 3D scene reconstruction. The method represents scenes as collections of 3D Gaussians rather than implicit neural fields, enabling fast rendering. The post serves as an educational overview of the technique's mechanics and applications.
OrbitForge: Text-to-3D scene generation via reconstruction-anchored video synthesis using Gaussian Splatting
OrbitForge is a new method for converting text-generated videos into 3D Gaussian Splatting scenes without task-specific fine-tuning or score-distillation optimization. The approach uses a frozen video diffusion model as a prior, performs an initial 3D reconstruction via Deformable Gaussian Splatting, detects missing viewpoints from a prescribed orbit, and completes only those views before final reconstruction. On a 300-prompt T3Bench-derived audit, OrbitForge achieves a 359-degree median orbit span and substantially improves coverage quality over a MedianGS-only baseline. The work also argues for coverage-aware evaluation metrics in text-to-3D tasks.
FLUX3D: Diffusion-aligned sparse representation for high-fidelity image-to-3D Gaussian Splatting
Researchers introduce FLUX3D, an image-to-3D Gaussian Splatting framework that addresses two structural bottlenecks in sparse voxel-based 3D generation: a representation bottleneck from discriminative 2D features and a cross-modal correspondence bottleneck in diffusion transformers. The system introduces Diffusion-Aligned Structured Latents (DA-SLAT) and a Sparse-structure Multimodal Diffusion Transformer (SMDiT) with Modal-Aware Rotary Positional Embedding (MARoPE) to improve 2D-3D alignment. Benchmark results claim substantial improvements in appearance fidelity over all current state-of-the-art methods for 3DGS asset generation.
HAT-4D: Agentic framework for 4D multi-object interaction reconstruction from monocular video
HAT-4D is a new agentic framework that reconstructs 3D geometry, temporal dynamics, and physical interactions of multiple objects from single monocular videos, targeting scalable data collection for Embodied AI and Vision-Language-Action (VLA) model training. The system integrates VLMs with a multi-level human-in-the-loop feedback mechanism to resolve depth ambiguities and occlusions without expensive multi-camera rigs. The authors also introduce MVOIK-4D, an open-world benchmark for monocular 4D interaction reconstruction with a novel evaluation protocol focused on physical plausibility and temporal consistency. Experiments show state-of-the-art performance on most metrics, and HAT-4D-generated data improves downstream model fine-tuning.
Video generation models as world simulators
OpenAI introduces Sora, a large-scale text-conditional video diffusion model built on a transformer architecture that operates on spacetime patches of video and image latent codes. The model is trained jointly on videos and images of variable durations, resolutions, and aspect ratios. Sora can generate up to one minute of high-fidelity video and OpenAI frames scaling video generation as a path toward general-purpose physical world simulators.
PEVA: Whole-Body Conditioned Egocentric Video Prediction for Embodied World Models
Researchers from BAIR introduce PEVA (Predicting Ego-centric Video from human Actions), a model that generates first-person video frames conditioned on 48-dimensional whole-body kinematic pose trajectories. The model uses an autoregressive conditional diffusion transformer trained on the Nymeria dataset, which pairs real-world egocentric video with body pose capture. PEVA can generate atomic action videos, simulate counterfactuals, and support long video generation, representing a step toward world models grounded in physically embodied human agents.
Genie 3: A new frontier for world models
DeepMind has announced Genie 3, a world model capable of generating interactive, navigable 3D environments in real time at 24 fps and 720p resolution. The system maintains consistency for several minutes, representing a significant step up from prior Genie iterations. This positions Genie 3 as a frontier capability demonstration in generative world modeling for interactive applications.
IVGT: Implicit Visual Geometry Transformer for Neural Scene Representation
IVGT is a new neural architecture that implicitly models continuous 3D geometry from unposed multi-view images without requiring explicit pointmap regression. It learns a continuous neural scene representation in a canonical coordinate system, supporting SDF-based surface queries and color prediction via lightweight decoders. The model is trained with multi-dataset joint optimization using 2D supervision and 3D geometric regularization, achieving strong generalization across mesh reconstruction, novel view synthesis, depth/normal estimation, and camera pose estimation tasks.