For a clue to AMI Labs’ possible commercial future, consider a cook making pie crust.
An AI assistant follows the preparation, offering instructions as the work progresses. When the cook pauses for coffee, it recognizes the detour and keeps the recipe on track.
That demonstration captures the ambition running through Pascale Fung’s newly posted ICML talk from July, “Towards AI Agents in the Real World.” AMI’s co-founder and chief research and innovation officer wants agents that understand what people are doing, anticipate what comes next, and help when everyday activities get messy.
The clearest commercial clue concerns wearable assistants. Fung explicitly described agents in wearable devices as work she pursued at Meta FAIR and continues at AMI. Her examples centered on smart glasses sharing their wearer’s view and hearing: a platform for coaching, navigation, learning, and practical assistance.
“Nobody wants to be always asking the glasses what to do,” she said.
Imagine arriving at an unfamiliar Paris metro ticket machine. The assistant could guide you through buying a ticket, or complete the purchase digitally on your phone. With persistent memory, it would also remember that you had mastered the machine, sparing you another tutorial on the next visit.
These scenarios make AMI’s interest in world models tangible. Fung argued that today’s language models learn about reality secondhand, through descriptions, and remain unreliable at understanding physical consequences. A system directing a robot or helping someone perform a task needs to track what actually happened, including mistakes, and adjust its plan accordingly.
Her presentation linked two research approaches. A Vision-Language World Model plans sequences of actions and their consequences using language descriptions grounded in visual observations. A separate model, VL-JEPA, tracks actions and changing conditions through compact mathematical representations and producespower-limited devices text only when needed. Together, planning and observation let an assistant keep pace with an unfolding activity.
Efficiency is central to that proposition. Fung highlighted VL-JEPA results showing stronger performance with 50% fewer trainable parameters in a controlled comparison against conventional vision-language training. Selective decoding also reduced text-decoding operations by a factor of 2.85 compared with decoding at fixed intervals while preserving similar performance. Such savings could matter for assistants expected to respond immediately on devices with limited power.
Robotics offers another potential application. Fung described a bet on a general world model that can be adapted to different tasks, with planning guided by the desired outcome. The ambition is to escape the brittleness of robots whose learned routines break when their surroundings change. Wearable assistants and autonomous machines would require different levels of control, but both need reliable predictions about actions and consequences.
Then comes the human complication. Fung argued that agents must model the “mental world” alongside the physical one: intentions, attention, preferences, and social context. An assistant should distinguish an experienced cappuccino maker from someone encountering the machine for the first time. Knowing when to speak, and when to stay quiet, becomes part of its usefulness.
She also acknowledged the risks. Always-on cameras and microphones capture bystanders as well as users; processing on the device could reduce exposure. Humanlike voices and behavior can encourage misplaced trust or emotional attachment. Those concerns follow directly from the intimate, continuous assistance she envisions.
For AMI watchers, wearable AI is therefore a concrete application area to follow. The talk combines earlier research with an agenda Fung explicitly connects to her current work. It does not announce AMI-branded glasses, a commercial partner, or a launch timetable. What it does offer is a picture of potential usefulness: an assistant that remembers your experience, notices your mistakes, and knows a coffee break has not ruined the recipe.