A new research paper featuring contributions from AMI researchers shows that high-quality video representations can be learned with a radically simpler JEPA-style system — one encoder, one objective, and no target network, predictor or stop-gradient — while matching or beating V-JEPA 2 with as much as 20.8× less pretraining compute and producing causal representations suited to future world models.
The central contribution of LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics is to remove much of the machinery that has made self-supervised video learning expensive and complicated. Existing approaches such as V-JEPA rely on architectural tricks — including a separate target encoder, stop-gradient operations, and a predictor — to stop representations from collapsing to meaningless constants. LeVJEPA instead imports the SIGReg regularization developed for LeJEPA, explicitly constraining the learned embedding distribution so that collapse is excluded with a theoretical guarantee. The result is an unusually clean recipe: the trainable system consists essentially of an encoder and a small projector, with a single fixed hyperparameter.
That simplification produces surprisingly large efficiency gains. On identical data and matched training epochs, LeVJEPA matches or exceeds V-JEPA 2 across small, base, and large Vision Transformers while requiring 5.6× to 20.8× less total pretraining compute. Even more strikingly, the researchers find that randomly dropping 95% of the video patch tokens during training not only reduces computation but also improves ImageNet performance, suggesting token dropping functions as a powerful augmentation rather than merely a cost-saving approximation. At equal compute, LeVJEPA reaches 61.0% on ImageNet-1K versus 53.4% for VideoMAEv2 and 51.6% for V-JEPA 2, while also posting the strongest Kinetics-400 result of the three. A small version could even be pretrained for 12 hours on a single RTX 5080, substantially lowering the hardware barrier to video representation learning.
The potentially bigger implication for world models is causality. LeVJEPA can use block-causal attention with essentially no accuracy penalty, meaning each frame’s representation depends only on that frame and what came before it, rather than seeing the future. That matters because streaming perception and autoregressive world models need exactly this property: as a new frame arrives, the system can extend its internal representation instead of reprocessing the entire video history. The paper therefore suggests that video may become a practical general-purpose pretraining substrate rather than an expensive specialist alternative to images — especially since, at matched compute, LeVJEPA comes close to DINOv2 on appearance while achieving nearly twice its motion-centric accuracy.
The work includes several AMI Labs-affiliated researchers: Quentin Le Lidec, Lucas Maes, and Yann LeCun. Once again, Brown University’s Randall Balestriero is identified as a member of AMI Labs.