Your brain has 4 types of memory. So does your AI agent. Here's which one to give it.

Introduction

An AI agent repeating the same mistake twice, forgetting what was told 10 messages ago, or slowing down with too many instructions is not a bug. It is a memory problem with a precise solution.

Princeton researchers formalized in 2023 (CoALA framework) a memory model for AI agents directly inspired by human cognition. Four types, four distinct roles, and one simple rule: each agent type only needs the memory that matches its tasks.

The human parallel

Neuroscientists distinguish four major memory systems in humans:

• Working memory – what you hold in mind during a conversation
• Semantic memory – facts and concepts you know
• Procedural memory – how you do things without thinking about them
• Episodic memory – personal memories with their context

AI agents have exactly the same four types. This is an architecture, not a metaphor.

Type 1 – Working Memory

Human analogy: Your computer’s RAM.

Everything the agent “sees” during a session lives here: messages, conversation history, system instructions, tool results. When the session ends, everything disappears.

The problem: this memory has a hard size limit – the context window. Fill it too fast and the agent starts “forgetting” older elements or degrading in performance.

What this means for you: every instruction in the system prompt, every tool call, every returned result consumes working memory. Managing an agent’s memory starts with managing this budget.

Type 2 – Semantic Memory

Human analogy: Your personal knowledge base. Everything you know, without necessarily remembering how you learned it.

For an agent, this is typically a file loaded at the start of each session: claude.md, AGENT_CONTEXT.md, or a config document. It holds stable facts:

• Who the user is
• What the project context is
• What the business rules are
• What tools are available

What this means for you: if your agent asks the same questions every session, that information is not in semantic memory. Put it in a context file loaded at startup.

Type 3 – Procedural Memory

Human analogy: Knowing how to drive – automatic, not thought about.

For an agent, this is its set of skills: detailed instructions on how to execute complex tasks (how to run a security audit, how to write a report, how to call an API).

The key principle is progressive disclosure. Do not load all skills into working memory upfront – that kills the context window. Instead, the agent has a lightweight index (name and short description for each skill). When a task matches a specific skill, the agent loads the full instructions.

What this means for you: if you are building a multi-task agent, do not dump all instructions into the system prompt. Create separate skill files and a conditional loading mechanism.

Skill Index
 ├─ Security Audit
 ├─ API Calling
 ├─ Reporting
      ↓
Load only when needed

Type 4 – Episodic Memory

Human analogy: Personal memories – not just “bugs often happen on Fridays” (semantic), but “on Friday March 14th, we debugged this module for 45 minutes and the problem was an unhandled network timeout.”

For an agent, this is the record of past sessions with enough context to be useful next time.

The golden rule is distillation. Do not save raw transcripts. What matters is the distilled extract:

“Session 03/14 – Payment module. Issue: unhandled network timeout on Stripe API call. Fix: increase timeout to 30s and add retry with exponential backoff.”

A 200-message transcript is noise. Do not summarize too early or too aggressively – overly aggressive summarization turns distinct episodes into semantic generalizations, losing the specific context that makes the memory valuable.

Repeated patterns in episodic memory eventually feed into semantic memory. What was a memory becomes general knowledge – continuous learning.

Which agent needs what?

Agent type	Working	Semantic	Procedural	Episodic
Simple reflex (thermostat, spam filter)	✔	✖	✖	✖
Customer support	✔	✔	✔	◐ Optional
Autonomous / long-term agent	✔	✔	✔	✔

• Simple reflex agent: perceives, reacts, forgets. No persistent memory needed.
• Customer support agent: needs procedures, product context, and optionally past interactions.
• Autonomous agent: operates over weeks or months. Needs everything, including well-managed episodic memory to avoid reinventing the wheel.

The takeaway

An AI agent’s memory is not a technical detail – it is its cognitive architecture. Design it poorly and you get an agent that forgets, repeats itself, or burns through its context window.

Design it well and you get an agent that:

• Knows its context without being told every time (semantic)
• Executes without over-thinking (procedural + progressive disclosure)
• Learns from past sessions without storing everything (episodic + distillation)
• Stays sharp in the current conversation (well-managed working memory)

Next time you design an agent, ask yourself before dumping everything into the system prompt: which memory, for which role?