Household robots will not become truly useful simply by recognizing objects or following one-off commands. They need memory: the ability to connect what they saw earlier with what they are being asked to do now. A new benchmark called FindingDory is helping researchers test ai memory for household robots by measuring whether vision-language models can recall relevant visual experience, reason over it, and use it to guide actions in simulated home environments.
Why does memory matter for household robots?
Memory matters because a robot in a home is not solving a single isolated puzzle; it is operating in a changing, personal space. If it tidies a room, moves keys from a table, or notices that a bag was left near the door, it should be able to explain where things went and act on what it learned later. Georgia Tech researchers describe this as a core challenge for in-home assistive robots: without stronger memory, robots could create inconvenience instead of reducing it.
That is different from a typical smart speaker interaction. Many ai home assistants can answer a question, set a timer, or trigger a connected device, but a mobile household robot must remember visual details across time and space. It has to connect “the diary on the nightstand yesterday” with “bring me the diary now,” even if the object appears in only a few frames among many observations.
The new benchmark focuses on embodied memory
FindingDory was created to evaluate memory in embodied agents, meaning AI systems that perceive and act in an environment rather than only process text on a screen. The benchmark uses the Habitat simulator and includes 60 tasks that require sustained engagement and contextual awareness. Its tasks are designed around practical embodied challenges such as navigation, object interaction, and reasoning over earlier observations.
The project is especially relevant because vision-language models, or VLMs, are increasingly being explored for robotics. These models can connect images and language, which makes them promising for home automation ai and household robotics. But the researchers note that current VLMs typically struggle to process more than a few hundred images at once, while a robot at home may collect long streams of visual experience over hours, days, or longer.
In plain English, the issue is not only whether an AI can “see” a mug. The harder question is whether it can remember where the mug was, what happened to it, which room it appeared in, and whether that information is relevant to the current request.
What the benchmark actually tests
FindingDory evaluates three broad kinds of memory-heavy tasks: spatial, temporal, and multi-goal reasoning. These categories matter because real homes are full of partial information. A robot may not observe everything continuously, and a useful AI memory system has to decide what to keep, what to ignore, and how to retrieve the right detail later.
The benchmark includes prompts that require the agent to reason over prior experience, such as navigating to an object or room based on what happened the previous day. According to coverage of the work, its evaluations average roughly a one-day memory period, which is already demanding for systems that must reason over visual histories rather than compact text logs.
Key abilities being tested include:
- Spatial recall: remembering where an object, room, or receptacle appeared within a home-like layout.
- Temporal awareness: distinguishing what was seen earlier from what is being observed now.
- Relevant retrieval: finding the few important moments inside a large stream of visual data.
- Action grounding: using remembered information to complete a navigation, pick, or place instruction.
- Multi-goal reasoning: completing sequential goals without losing track of previous steps.
This is why ai memory systems for robots are harder than chat history for a chatbot. Text is comparatively compact, while video is dense, repetitive, and full of distracting detail. A useful robot memory layer needs to compress experience without deleting the moment that becomes important later.
Current models still struggle
The early results are a reminder that impressive AI demos do not automatically translate into reliable household autonomy. The FindingDory project page reports that proprietary VLMs such as Gemini 2.0 Flash and GPT-4o fail to perform strongly across most task categories, with multi-goal tasks creating particular difficulty. The best-performing baseline cited by the project was a Qwen-SFT model, yet it still reached only about 50% on high-level tasks.
That result should not be read as a failure of robotics research. It is more useful as a map of the gap. Models can recognize objects, describe scenes, and follow language instructions, but home robots need persistent, selective, action-ready memory. The difference between “I saw a wallet” and “I placed your wallet in the blue bowl near the entryway after clearing the table” is the difference between novelty and dependable assistance.
The benchmark also suggests that simply feeding more frames to a model is not a complete solution. The project page notes that frozen VLMs did not improve just because they received more frames, while a fine-tuned Qwen model benefited more clearly from longer videos. That points toward a future where memory architecture, training, retrieval, and reasoning all matter.
Better robot memory will change smart homes
If ai memory for household robots improves, the impact could be felt across the broader smart home ecosystem. Today, many smart home products rely on rules: turn on the lights at 7 p.m., start the vacuum when nobody is home, lock the door when the phone leaves a geofence. A robot with stronger memory could make home automation ai more contextual.
Instead of only executing commands, it could learn household routines and respond to exceptions. For example, it might notice that shoes are usually stored by the door but today are in the hallway, or that a medication bottle was moved from the kitchen counter to a bathroom shelf. Those examples are not claims about a commercial product; they illustrate the kind of memory-dependent reasoning researchers are trying to measure.
For ai home assistants, this also raises the bar. Voice assistants are helpful when a user knows what to ask and the system has access to the right data. A memory-capable embodied assistant would add physical context: what was seen, where it was seen, and how that relates to the user’s present intent.
What should builders learn from this benchmark?
Builders should treat memory as a product requirement, not an optional feature added after navigation or manipulation. FindingDory shows that long-horizon household tasks require both recall and execution: the robot must retrieve the relevant past observation and then act on it in the environment. The arXiv paper argues that memory integration should be studied together with action because embodied agents need both to succeed.
For teams working on domestic robots, smart home platforms, or AI assistants, the practical lessons are clear:
- Design for selective memory. A home robot cannot keep every frame equally available forever. It needs ways to identify important objects, events, rooms, and user preferences.
- Separate short-term and long-term context. Immediate tasks may need recent observations, while personalization depends on durable memory that survives across sessions.
- Make memory explainable. Users will want to know why a robot moved an object or how it inferred a preference.
- Plan for correction. If a user says, “That does not belong there,” the system should update future behavior rather than repeat the same mistake.
- Protect privacy from the start. Memory inside a home involves sensitive visual and behavioral data, so storage, deletion, and user control must be part of the design.
The most useful ai memory systems will likely be judged not only by benchmark scores but also by how gracefully they handle everyday ambiguity.
The path forward for household robotics
The researchers behind FindingDory suggest potential routes for improvement, including converting visual experience into descriptive text and using hierarchy to remove irrelevant or redundant images from storage. That makes intuitive sense: a robot does not need to remember every pixel of a quiet hallway, but it may need a compact note that the red backpack was moved from the hallway to the closet.
At the same time, compression can be risky. If the robot summarizes too aggressively, it may discard the exact detail a user later needs. The challenge is to preserve useful memory while keeping storage and retrieval manageable.
A practical future robot memory stack may combine several layers:
- recent sensory context for immediate actions;
- object and location memory for household organization;
- episodic memory for notable events and user corrections;
- preference memory for repeated routines;
- safety and privacy rules that limit what can be stored or shared.
This layered approach would bring household robots closer to the way people use memory: not as a perfect recording, but as a structured, retrievable understanding of what matters.
A useful reality check for AI at home
FindingDory is important because it tests a capability people will expect from household robots long before the technology is perfect. If a robot cleans, organizes, or assists around the home, users will naturally assume it remembers what it did. The benchmark shows that current systems still have significant work ahead before that expectation can be met reliably.
For now, the key takeaway is simple: the next leap in home robotics may depend less on flashy movement and more on dependable memory. Better ai memory for household robots could turn smart devices from reactive tools into assistants that understand context, history, and the small details that make a home personal.
