Memory in AI Agents: How Agents Remember, Retrieve, and Forget

Memory is not a magical property of the model. It is a system the agent uses to preserve, retrieve, and revise useful information over time.

A language model can generate a fluent answer from the information placed in front of it. But when the request ends, the model does not automatically carry a private record into the next request. An AI agent becomes capable of continuity only when the surrounding system deliberately manages memory.

That distinction matters. A useful agent may need to remember that a customer prefers email, that a refund has already been approved, that a deployment failed yesterday, or that a task is waiting for human approval. Without memory, every interaction begins close to zero. With careless memory, the agent can become confidently wrong, intrusive, or overwhelmed.

This lesson explains memory as part of the larger [anatomy of an AI agent](/anatomy-of-an-ai-agent/): what gets remembered, where it lives, how it returns to the model, and when it should be updated or forgotten.

TL;DR

• The context window is the information available to the model for one inference; memory is the broader system that preserves and recalls information across steps or sessions.
• Working memory keeps the current goal, observations, intermediate results, and active constraints close at hand.
• Long-term memory can store experiences, facts, user preferences, summaries, and reusable knowledge outside the model.
• Retrieval is as important as storage. A memory that cannot be found at the right moment is functionally absent.
• More memory is not automatically better. Good memory is relevant, current, attributable, permissioned, and easy to correct or delete.

Why an Agent Needs Memory

Consider a travel-planning agent used over three conversations.

Interaction 1: You say you are based in Beijing, prefer morning flights, and avoid overnight connections.

Interaction 2: A week later, you ask for routes to Singapore. The agent should reuse the stable preferences but check whether they still apply to this trip.

Interaction 3: You approve one itinerary and ask the agent to prepare a booking checklist. It must remember which itinerary was selected, not merely the options it previously suggested.

Three different kinds of information are involved:

• A user preference that may remain useful across future trips
• A past episode describing what happened in a previous conversation
• A current task state recording which option has been selected

Storing all three as an undifferentiated transcript would be easy. Using them reliably would be hard. Agent memory exists to turn raw history into useful continuity.

This extends the practical model introduced in [What Is an AI Agent?](/what-is-an-ai-agent/) and the execution cycle described in [How AI Agents Work](/how-ai-agents-work/). The agent does not only reason, act, and observe. It also decides what observations deserve to survive beyond the current step.

Memory Starts With a Stateless Model

At the model level, each inference is driven by an input. That input may include system instructions, the current user message, selected conversation history, retrieved documents, tool results, and other application data.

The model can use only what is available through that input and what its trained parameters already encode. The application may make a conversation feel continuous by repeatedly sending earlier information or retrieving stored records, but that continuity is created by the agent system.

This is why agent memory is best understood as an architecture, not a personality trait.

The CoALA framework for language agents makes a useful distinction: working memory holds active information for the current decision cycle, while long-term memory can include episodic, semantic, and procedural forms. It also treats retrieval and memory writing as internal actions performed by the agent system.

Context Windows: The Agent’s Active Workspace

A context window is the bounded amount of input and output information a model can reference during one inference. It is often measured in tokens.

The context may contain:

• Instructions and safety rules
• The current goal
• Recent conversation turns
• Tool definitions and tool results
• Retrieved memories or documents
• A plan or task summary
• The response being generated

The context window is temporary and capacity-limited. Even when a model supports a large window, filling it with every available transcript, document, and tool result can increase cost and bury the relevant signal. Research on long-context models has also shown that information position can affect whether a model uses it successfully.

A practical agent therefore treats context as a curated workspace, not a storage warehouse.

Working Memory

Working memory is the small set of information the agent actively needs for its current decision cycle. It can include:

• Current goal: “Prepare the approved Singapore itinerary”
• Constraints: “Morning departure; no overnight connection”
• Recent observation: “The selected fare no longer includes checked baggage”
• Intermediate result: “Two alternatives remain”
• Pending action: “Wait for user approval before booking”

Some working memory is represented directly in the prompt. Some may exist as structured fields outside the prompt and be selectively inserted when needed.

Working memory is short-lived, but it does not have to disappear after one model call. The orchestration layer can carry structured variables from one step of the [agent loop](/how-ai-agents-work/) to the next.

Conversation History Is Not the Same as Memory

Conversation history is a record of messages. It is one possible source of memory, but it is not automatically a well-designed memory system.

A raw transcript contains:

• Stable facts
• Temporary requests
• Corrections
• Repeated information
• Abandoned ideas
• Sensitive details
• Assistant mistakes

If the agent treats every sentence as equally durable, it will eventually retrieve noise or preserve something the user never intended to make permanent.

A better system can extract a compact fact such as “User generally prefers morning flights,” store its source and timestamp, and keep the original conversation available only when policy and need justify it.

Short-Term vs Long-Term Memory

Dimension	Short-term memory	Long-term memory
Main purpose	Support the current interaction or task	Preserve useful information across sessions or tasks
Typical contents	Recent turns, current goal, active plan, latest observations	Preferences, past outcomes, learned facts, durable summaries
Lifetime	Seconds, minutes, or one task	Days, months, or policy-defined retention period
Common location	Prompt, runtime object, session store	Database, document store, vector store, knowledge graph
Main risk	Losing an active constraint	Preserving stale, incorrect, or sensitive information

The boundary is architectural rather than biological. A session summary stored for 24 hours may act as short-term memory in one product and long-term memory in another.

Types of Long-Term Memory

Episodic Memory: What Happened

Episodic memory records specific experiences or events:

• “On July 29, the user rejected the overnight itinerary.”
• “The deployment failed after the database migration.”
• “The customer accepted a replacement rather than a refund.”

Episodes are useful when the agent must learn from prior attempts, maintain continuity, or explain how a decision was reached. They should normally include time, source, actors, and outcome.

Semantic Memory: What Is Known

Semantic memory stores facts and generalized knowledge:

• “The user’s default departure city is Beijing.”
• “The company’s refund window is 14 days.”
• “Project Atlas uses PostgreSQL.”

Some semantic memory comes from user interactions. Some comes from approved documents or databases. A useful semantic record should distinguish an observed fact from an inferred summary and should retain provenance.

User or Profile Memory: What Helps Personalization

User memory is a practical subset of semantic and episodic memory focused on the person:

• Preferences
• Recurring constraints
• Role or expertise
• Explicitly saved goals
• Prior decisions relevant to future assistance

Profile memory should not become unrestricted surveillance. The product needs clear controls for what can be stored, how long it remains, where it is used, and how the user can inspect, correct, or delete it.

Procedural Memory: How to Do Something

Some architectures also describe reusable procedures, skills, or successful action sequences as procedural memory. For example, a coding agent might retain an approved deployment checklist or a tested recovery procedure.

This lesson focuses mainly on working, episodic, semantic, and user memory, but procedural memory becomes important when agents learn repeatable ways of acting.

The Memory Lifecycle

Agent memory should be designed as a lifecycle:

Observe → Decide What Matters → Store → Retrieve When Relevant → Use → Update or Forget

Each stage is a quality gate.

1. Observe

The agent receives a user message, tool result, environmental signal, or task outcome.

Example: “For business trips, I prefer an aisle seat.”

2. Decide What Matters

The system asks whether the information is durable, useful, safe, and permitted to store.

The aisle-seat preference may be reusable. A one-time request such as “choose the 7:00 flight today” probably belongs in task state, not permanent profile memory.

3. Store

The system writes a structured memory with useful metadata:

{
  "subject": "user",
  "type": "travel_preference",
  "fact": "Prefers an aisle seat for business trips",
  "scope": "business_travel",
  "source": "user_explicit",
  "recorded_at": "2026-07-30",
  "confidence": 1.0
}

The stored form should be compact enough to retrieve and precise enough to avoid overgeneralization.

4. Retrieve When Relevant

When a later request concerns a business flight, the system searches for relevant memories. Retrieval can combine:

• Exact filters, such as user ID, memory type, project, or date
• Keyword search
• Semantic similarity
• Recency
• Importance
• Confidence or trust level

Retrieval should produce a small, ranked candidate set—not the entire memory archive.

5. Use

The selected memory is inserted into working context and influences reasoning or action.

The agent might say: “I’ll prioritize aisle seats, based on your saved business-travel preference.”

Making the use visible is often valuable because the user can confirm or correct it.

6. Update or Forget

If the user says, “I now prefer window seats,” the agent should update or supersede the old record rather than preserve two equal, conflicting truths.

Forgetting may mean:

• Deleting a memory
• Expiring it after a retention period
• Reducing its retrieval priority
• Marking it superseded
• Aggregating old events into a summary
• Keeping it for audit but excluding it from normal retrieval

Forgetting is not a defect. It is part of memory quality.

External Memory Stores

Persistent agent memory usually lives outside the language model. Common storage choices include:

• Relational databases for structured facts, timestamps, ownership, and updates
• Document stores for summaries, notes, and semi-structured records
• Vector databases or vector indexes for semantic similarity search
• Knowledge graphs for explicit entities and relationships
• Event logs for ordered actions, observations, and outcomes

Many production systems combine them. A relational database may hold the authoritative preference and permissions, while a vector index helps find related past episodes.

The storage technology is not the memory itself. The memory system also needs policies for writing, ranking, conflict resolution, retention, and access control.

Embeddings and Vector Retrieval

An embedding represents content as a numerical vector. Texts with related meanings can be placed near one another in that vector space.

A simple semantic retrieval flow is:

1. Convert stored memory text into embeddings.
2. Store the vectors with the original records and metadata.
3. Convert the new query into an embedding.
4. Search for nearby vectors.
5. Filter and rerank the candidates.
6. Insert only the best memories into context.

Vector search is useful when the wording changes. “I don’t want late connections” may still retrieve “User avoids overnight layovers.”

But similarity is not truth. A nearby record can be outdated, belong to the wrong project, or conflict with an authoritative source. Metadata filters, timestamps, provenance, and structured checks remain necessary.

Memory vs Context, State, RAG, and Model Knowledge

Memory stores potentially reusable information; retrieval selects some of it into context, while state tracks the current task and model knowledge remains encoded in parameters.

Concept	What it is	Example
Context	Information supplied to the model for the current inference	Current prompt, recent turns, retrieved memory
Memory	Information preserved so it can influence future steps or sessions	Saved preference or past task outcome
Agent state	The authoritative snapshot of the current task or workflow	approval_status = pending
RAG	A pattern that retrieves external knowledge to augment generation	Fetching policy passages before answering
Model knowledge	Patterns and information encoded in trained parameters	General language and broad world knowledge

Memory vs Context

Memory is what may be available later. Context is what is actually placed before the model now.

A stored preference has no effect until the system retrieves it and puts a usable representation into context.

Memory vs Agent State

Memory supports recall; state tracks where the task currently stands.

For a refund agent:

• “Customer prefers store credit” can be memory.
• “Refund request #482 is awaiting manager approval” is task state.

State should usually be structured, authoritative, and transactionally updated. Treating critical workflow state as fuzzy prose memory can cause duplicate or unsafe actions.

Memory vs RAG

RAG is a retrieval-and-generation pattern, not a synonym for agent memory.

A RAG system may retrieve static product manuals without remembering anything about the user or its own past actions. An agent memory system may retrieve a user preference without answering a knowledge question.

The two can share embeddings, vector indexes, and retrieval pipelines. Their purpose and governance are different.

Short-Term vs Long-Term Memory

Short-term memory prioritizes immediate continuity. Long-term memory prioritizes future reuse. Moving information from short-term to long-term should be a deliberate write decision, not an automatic transcript dump.

Model Knowledge vs Agent Memory

Model knowledge is learned during training and is difficult for an application to inspect or update precisely. Agent memory is application-managed, can be sourced from recent interactions, and should be correctable or deletable.

The model may know what an aisle seat is. The agent memory records that this user prefers one.

Memory Is Not Reasoning

Memory provides information; reasoning transforms information into judgments or decisions.

Retrieving “the user avoids overnight connections” does not decide which flight to choose. The agent must combine that memory with price, schedule, and the current goal. That decision belongs to [reasoning in AI agents](/reasoning-in-ai-agents/).

Likewise, a memory system may use [tool calls](/tool-use-in-ai-agents/) to query a database, but the tool is the access mechanism—not the remembered information.

Common Failure Modes

Irrelevant Memories

Semantic search returns something vaguely similar but unhelpful. The model then anchors on it.

Mitigation: combine similarity with metadata filters, recency, task scope, and minimum relevance thresholds.

Stale Memories

A preference, project fact, or policy changes, but the older record remains highly ranked.

Mitigation: store timestamps and validity periods; favor authoritative sources; expire, supersede, or revalidate time-sensitive records.

Conflicting Memories

The system stores both “prefers morning flights” and “prefers evening flights” without context.

Mitigation: preserve scope and source, detect conflicts during writes, ask for clarification, and mark older records as superseded instead of silently deleting provenance.

Excessive Memory

The agent saves everything and retrieves too much. Costs increase while relevance falls.

Mitigation: use explicit write criteria, summaries, deduplication, retention limits, and a strict context budget.

False or Overgeneralized Memory

The agent converts “I need a morning flight this Tuesday” into “The user always prefers morning flights.”

Mitigation: distinguish one-time constraints from stable preferences; preserve exact wording or provenance; use confidence and scope fields; allow confirmation before durable storage.

Poor Retrieval

The correct memory exists but is not selected because the query, embedding, chunking, filters, or ranking logic is weak.

Mitigation: evaluate retrieval separately from answer quality using known relevant memories, hard negatives, ranking metrics, and end-to-end task tests.

Privacy and Permission Failures

The system stores sensitive information without a valid purpose, exposes one user’s memory to another, or makes deletion difficult.

Mitigation: minimize collection, isolate tenants, encrypt sensitive stores, enforce access controls, log memory access, set retention rules, and give users meaningful inspect/correct/delete controls.

What Makes Memory High Quality?

A useful memory should be:

• Relevant: likely to help a future decision
• Specific: scoped to the right user, task, project, or situation
• Current: timestamped and updated when the world changes
• Grounded: linked to a source rather than generated from guesswork
• Non-duplicative: consolidated with equivalent records
• Permissioned: stored and retrieved under clear access rules
• Controllable: correctable, exportable, and deletable where appropriate
• Retrievable: indexed and ranked so it appears at the right moment

Memory quality should be evaluated at multiple points:

1. Write precision: Did the agent save information worth keeping?
2. Write recall: Did it miss information that should have been saved?
3. Retrieval relevance: Did the right memories rank highly?
4. Freshness: Were stale or superseded records suppressed?
5. Usefulness: Did the retrieved memory improve the task outcome?
6. Safety: Did storage and retrieval respect policy and user expectations?

An agent can have an excellent language model and still fail because its memory pipeline is weak.

Builder and Product Implications

For a first memory-enabled agent, keep the architecture conservative:

1. Define a small set of memory types.
2. Separate task state from long-term memory.
3. Require clear write criteria.
4. Store source, scope, timestamp, confidence, and ownership.
5. Retrieve a few candidates using filters plus semantic or keyword search.
6. Make memory use visible when it affects a decision.
7. Provide correction, deletion, and retention controls.
8. Evaluate writes and retrieval independently.

Do not begin with “remember everything.” Begin with one high-value continuity problem, such as saved preferences, past task outcomes, or handoff summaries.

My Take

The hardest part of agent memory is not storage capacity. It is judgment.

The system must decide what deserves to persist, which record should win when facts conflict, when a memory is relevant enough to surface, and when forgetting is safer than recall. Those are product, data, and governance decisions as much as model decisions.

A strong memory system behaves less like an infinite transcript and more like a well-maintained working notebook: selective, sourced, organized, revisable, and under the user’s control.

Next Step

Memory gives an agent continuity. Planning gives it a way to organize future action.

Continue with Start Here Lesson 07: Planning in AI Agents after it is published, or review the complete learning path on the [Start Here page](/start-here/).

Sources

• Sumers et al., Cognitive Architectures for Language Agents (CoALA)
• Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Park et al., Generative Agents: Interactive Simulacra of Human Behavior
• Liu et al., Lost in the Middle: How Language Models Use Long Contexts
• OpenAI, Retrieval guide: semantic search and vector stores
• Anthropic, Context windows