Research

This work proposes CrossBERT, a two-part architecture that separates the learning of high-quality encoded representations from the rigid grounding of token reconstruction, and demonstrates monotonic scaling and superior performance on MTEB(eng, v2) and frozen GLUE benchmarks.

This work proposes Anchored Self-Play (ASP), which anchors self-play with a small reference set by adding a code-embedding similarity reward for generation and mixing reference bugs into fixer training, and achieves the best fix rates across bug sources.

This study evaluates the performance of lightweight LLMs (under 8B parameters) in generating biomedical abstracts and lay summaries in a zero-shot setting and introduces a novel analysis of the sectional origin and desirability of information.

C-JEPA is a simple and flexible object-centric world model that extends masked joint embedding prediction from image patches to object-centric representations and induces latent interventions with counterfactual-like effects and prevents shortcut solutions, making interaction reasoning essential.

This work introduces MJEPA, a joint-embedding predictive architecture for audio-visual learning that uses a single, unified encoder for both modalities, and shows that cross-modal prediction is critical: without it, a shared encoder degrades below unimodal baselines; with it, each modality's representation benefits from the other.

This work provides the first provable analysis of successful internalization: for the task of learning parities, it is shown that a simplified one-layer transformer provably first learns the target with explicit CoT supervision and then internalizes the autoregressive generation as CoT tokens are progressively removed, learning to directly compute the parity.

Hyperball is a simple optimizer wrapper that improves learning rate transfer across widths and depths compared to decoupled weight decay, motivated by prior theory showing that training with weight decay leads to an equilibrium weight norm that only depends on the training hyperparameters.

Cambrian-P is revisited as a lightweight supervisory signal and Cambrian-P, a video MLLM augmented with per-frame learnable camera tokens and a pose regression head is introduced, which achieves substantial gains on spatial reasoning benchmarks and achieves state of the art streaming pose estimation on ScanNet.

Stable-worldmodel (swm) is presented, an open-source platform for standardized and reproducible world modeling research and evaluation that dramatically reduces research overhead and accelerates trustworthy progress toward reliable world models.

PEIRA is introduced, a non-contrastive SSL method with an explicit objective defined through the trace of the optimal linear regressor, which shows that its only stable equilibria are nontrivial global minimizers and recover the same canonical correlation subspaces, with regularization selecting the effective dimension.

KVerus is presented, a retrieval-augmented system for Verus-based Rust verification that can adapt to an evolving software environment and represents a significant step towards making formal verification a scalable and sustainable practice for modern, security-critical software.

The SFI-Bench is introduced, a video-based benchmark with over 1,500 expert-annotated questions derived from diverse egocentric indoor video scans that provides a diagnostic tool for measuring progress toward more cognitively capable and truly grounded multimodal agents.

This work derives a first-order loss for Sobolev learning of diffusion-based policies using both trajectories and feedback gains for gradient-based TO solvers that also provide feedback gains, andorporating first-order information enables predictions with fewer diffusion steps, reducing inference latency.

This work presents a systematic study of replay buffers for LLM post-training, formalizing the optimal design as a trade-off between staleness-induced variance, sample diversity and the high computational cost of generation.

HWM learns world models at multiple temporal scales within a shared latent space, so predictions from the long-horizon model serve as subgoals for the short-horizon model via latent matching, without task-specific rewards, skill learning, or hierarchical policies.

V-JEPA 2.1 is presented, a family of self-supervised models that learn dense, high-quality visual representations for both images and videos while retaining strong global scene understanding and world modeling and demonstrates strong performance on several challenging benchmarks.

LeWorldModel is introduced, the first JEPA that trains stably end-to-end from raw pixels using only two loss terms: a next-embedding prediction loss and a regularizer enforcing Gaussian-distributed latent embeddings.

The UniG2U-Bench is introduced, a comprehensive benchmark categorizing generation-to-understanding (G2U) evaluation into 7 regimes and 30 subtasks, requiring varying degrees of implicit or explicit visual transformations, highlighting the necessity for more diverse training data and novel paradigms to fully unlock the potential of unified multimodal modeling.

It is demonstrated that the MoE architecture harmonizes this scaling asymmetry by providing the high model capacity required by language while accommodating the data-intensive nature of vision, paving the way for truly unified multimodal models.

SparksRL is a novel high-performance RL training system that preserves bit-exact updates without dropping or quantizing information, designed for commodity-networked, loosely-coupled GPU resources.

C-JEPA is a simple and flexible object-centric world model that extends masked joint embedding prediction from image patches to object-centric representations, and provides a formal analysis demonstrating that object-level masking induces useful inductive bias by controlling observability.

Stable-worldmodel (SWM) is introduced, a modular, tested, and documented world-model research ecosystem that provides efficient data-collection tools, standardized environments, planning algorithms, and baseline implementations to support robustness and continual learning research.

Divergence Proximal Policy Optimization (DPPO) is proposed, which substitutes heuristic clipping with a more principled constraint based on a direct estimate of policy divergence (e.g., Total Variation or KL).

This library provides modular, self-contained implementations that illustrate how representation learning techniques developed for image-level self-supervised learning can transfer to video, where temporal dynamics add complexity, and ultimately to action-conditioned world models, where the model must additionally learn to predict the effects of control inputs.

This work systematically investigates rewards derived from the probability or log-probability of emitting the reference answer (or any other prompt continuation present in the data), finding that using the log-probability of the reference answer as the reward for chain-of-thought (CoT) learning is the only option that performs well in all setups.

On-Demand Communication (ODC) is proposed, which adapts PS into Fully Sharded Data Parallel (FSDP) by replacing collective all-gather and reduce-scatter with direct point-to-point communication and is a superior fit for the prevalent imbalanced workloads in LLM post-training.

SOAR: An asymmetric self-play framework that uses meta-RL to surface pedagogical signals, suggesting that the ability to generate useful stepping stones does not require the preexisting ability to solve the hard problems, paving a principled path to escape reasoning plateaus without additional curated data.

Scaling simplifies the framework, and RAE-based diffusion models demonstrate faster convergence and better generation quality, establishing RAEs as a simpler and stronger foundation than VAEs for large-scale T2I generation.

This document consolidates publicly reported technical details about Metas Llama 4 model family to provide a compact technical reference for researchers and practitioners who need precise, source-backed facts about Llama 4.

Training VL-JEPA on Action100M demonstrates consistent data-scaling improvements and strong zero-shot performance across diverse action recognition benchmarks, establishing Action100M as a new foundation for scalable research in video understanding and world modeling.

This work is able to train a controller that maps known actions to latent ones, allowing us to use latent actions as a universal interface and solve planning tasks with the authors' world model with similar performance as action-conditioned baselines.

This work characterize models from this family of methods as JEPA-WMs and investigates the technical choices that make algorithms from this class work, and proposes a model that outperforms two established baselines, DINO-WM and V-JEPA-2-AC, in both navigation and manipulation tasks.

Chatbots are a promising direction in the digitalization of language education, but their potential can only be realized with an integrated approach that combines technological innovations with the professional development of teachers.

This work proposes an approach to enhance planning with JEPA world models by shaping their representation space so that the negative goal-conditioned value function for a reaching cost in a given environment is approximated by a distance (or quasi-distance) between state embeddings.

GPT-5 is a unified system with a smart and fast model that answers most questions, a deeper reasoning model for harder problems, and a real-time router that quickly decides which model to use based on conversation type, complexity, tool needs, and explicit intent.

DexWM, a Dexterous Interaction World Model that predicts future latent states of the environment conditioned on past states and dexterous actions, is introduced and demonstrates strong zero-shot transfer to unseen skills on a Franka Panda arm with an Allegro gripper.

By learning to insert and denoise frames in a sequence, Flowception seamlessly integrates different tasks such as image-to-video generation and video interpolation, and allowing to learn the length of videos jointly with their content.

VL-JEPA, a vision-language model built on a Joint Embedding Predictive Architecture (JEPA), predicts continuous embeddings of the target texts and natively supports selective decoding that reduces the number of decoding operations by 2.85x while maintaining similar performance compared to non-adaptive uniform decoding.