What are the four types of AI agent memory?

Working memory is the context window the agent sees right now. Semantic memory is general knowledge such as project conventions. Procedural memory is know-how such as skills, and episodic memory is a record of past interactions and what was learned. The split comes from the CoALA framework.

Is CLAUDE.md a form of memory?

Yes. CLAUDE.md is a Markdown file in a project that Claude Code loads at the start of each session, so it acts as persistent memory. IBM's video classes it as semantic memory, though files like it often also hold procedures such as build commands.

What is progressive disclosure in agent skills?

It means an agent first sees only each skill's name and short description, then loads the full instructions when a task matches, and loads extra files or scripts only while running the skill. This keeps many skills available without filling the context window.

All video notes

Video notes · Agent memory

The Four Types of Memory Every AI Agent Needs

Video by IBM Technology on YouTube · 11 min · uploaded May 2026

Notes and analysis by BrainTube, written from the video. Not affiliated with IBM Technology. Watch on YouTube

In short

IBM's explainer maps the CoALA framework's four memory types onto today's agents: the context window as working memory, CLAUDE.md-style files as semantic memory, SKILL.md skills as procedural memory and distilled session notes as episodic memory. It is a clear, accurate starting taxonomy with good concrete examples. It blurs semantic and procedural memory, says little about retrieval once memory outgrows a file, and names forgetting as the hardest problem without offering any approach.

The argument

The video argues that memory is what separates an agent from a chatbot, and that agents need the same kinds of memory people have. Using CoALA (Cognitive Architectures for Language Agents) from a Princeton team, it names four types. Working memory is the context window, fast but volatile and limited. Semantic memory is general knowledge, which in practice is often plain Markdown loaded at session start. Procedural memory is know-how, packaged as skills that load only when needed. Episodic memory is a distilled record of past work, which lets an agent improve but raises hard questions about what to forget. Not every agent needs all four: a thermostat needs only working memory, a coding agent needs everything.

Key ideas, with timestamps

0:27
Starting from human memory
He sorts human memory into short-term memory, factual knowledge such as company security policies, learned skills such as writing backwards on glass, and personal experience. His example of the last is three hours spent debugging a Kubernetes cluster before noticing he was pointed at the wrong one, which he returns to at the end.
1:44
The CoALA framework
He credits the four-way split to CoALA, Cognitive Architectures for Language Agents, from a Princeton research team. The paper (2023) is a common reference point in agent design, so naming it gives viewers somewhere to go deeper.
2:09
Working memory is the context window
Everything the agent can see now, the conversation, system instructions and loaded files, is working memory, like RAM. Even windows of a million tokens or more have a ceiling, and he notes that models lose track of material buried in the middle of a long context. Every chatbot has this layer, so it is not what makes an agent.
3:25
Semantic memory is often a Markdown file
Papers describe semantic memory as vector databases or knowledge graphs, but he says many production agents use plain Markdown. His example is CLAUDE.md in a project root, holding architecture, coding conventions, build commands, preferred frameworks and things to avoid, loaded at the start of every session.
4:51
Procedural memory as skills
He describes the agent skills standard: a folder with a SKILL.md file describing what the skill does and how to do it, from building a slide deck to running a code review. Progressive disclosure keeps costs down. The agent sees only a name and description, roughly a hundred tokens per skill, and loads full instructions and referenced files only when a task matches.
6:36
Episodic memory as distilled experience
Saving every transcript counts as episodic memory but is rarely useful. Production systems have the agent write itself short notes it judges useful later, such as a past auth bug turning out to be in the middleware layer, instead of a 45-minute transcript. This is where memory starts to look like learning.
7:52
Forgetting is the hard part
He calls episodic memory the hardest type to get right: what to delete, when something becomes obsolete, and whether to keep old project memories when a user changes jobs. People forget naturally; for agents, he says, forgetting is an engineering problem.
8:22
Match memory to the agent
A simple reflex agent such as a thermostat or routing bot needs only working memory. A narrow support agent that resets passwords adds procedural memory for that one skill. A coding agent needs all four, including automatic memory that learns across sessions.

What it gets right

Concrete mappings. Most taxonomies of agent memory stay abstract. Tying each type to a file or mechanism people already use (the context window, CLAUDE.md, SKILL.md, session notes) makes the framework something you can check against your own setup.

Plain files are honest. Saying that much production semantic memory is just Markdown is accurate and useful. Many teams reach for a vector database before they need one.

Distill, do not dump. The point at 7:03 is the right default for episodic memory. A one-line lesson is cheaper to load and easier to verify than a transcript, and it is what actually changes the next session.

Not every agent needs everything. The closing examples are a good guard against over-building. A password-reset bot with episodic memory is mostly a privacy liability.

Where it blurs or stops short

CLAUDE.md is not purely semantic. Build commands and a list of things not to do are know-how, which by his own definition is procedural. Real memory files mix types, and that matters for design: facts can sit in context permanently, but procedures are better loaded on demand, as skills are.

CoALA is a bit richer than shown. In the paper, working memory is one part and episodic, semantic and procedural memory are the long-term stores, with procedural memory including the agent's own code and the model's weights, not only written instructions. The video's version is a fair simplification, but skills are one form of procedural memory, not the whole of it.

Retrieval is missing. He contrasts skills, loaded on demand, with semantic memory, always present in context. That only holds while semantic memory fits in a file. Once an agent's knowledge base is large, it has to be searched, and how that search works (by keyword, by meaning, or both) decides whether the memory is useful.

No answer on forgetting. Naming the problem is fine for ten minutes, but some practical options exist: timestamp every memory, prefer newer entries when they conflict, scope memories to a project so they expire with it, and let users see and delete what was saved.

Who writes the memory? The video does not separate memory the agent writes about itself from memory a person curates. Agent-written notes can be wrong and then repeated in every later session, so some review step is worth having.

How this applies to memory across AI tools

All four types in the video live inside one agent. For a person, the semantic layer is often the one that matters most and the one that is most scattered: what you have read and watched sits in different apps, and each assistant only knows what you paste in. That layer is also the easiest to share, because knowledge does not depend on which agent is running.

BrainTube is one way to build it once: it saves YouTube videos with their transcripts, articles, PDFs and podcasts into one compiled knowledge base and exposes it as an MCP server, so Claude, ChatGPT, Cursor and other MCP clients can search it.

For the difference between in-context and stored memory, see persistent memory versus the context window. For memory in coding agents specifically, see Claude Code memory tools.

Who should watch it

Watch it if you are new to agent design and want vocabulary for the kinds of memory an agent can have; it is short enough to watch whole. The most useful parts are 3:25 to 6:30 (CLAUDE.md and skills with progressive disclosure) and 7:52 to 9:45 (forgetting and matching memory to the agent). Skip it if you already know CoALA or have built agent memory before, since there are no implementation details, code or numbers beyond the hundred-tokens-per-skill estimate.

Frequently asked

What are the four types of AI agent memory?: Working memory is the context window the agent sees right now. Semantic memory is general knowledge such as project conventions. Procedural memory is know-how such as skills, and episodic memory is a record of past interactions and what was learned. The split comes from the CoALA framework.
Is CLAUDE.md a form of memory?: Yes. CLAUDE.md is a Markdown file in a project that Claude Code loads at the start of each session, so it acts as persistent memory. IBM's video classes it as semantic memory, though files like it often also hold procedures such as build commands.
What is progressive disclosure in agent skills?: It means an agent first sees only each skill's name and short description, then loads the full instructions when a task matches, and loads extra files or scripts only while running the skill. This keeps many skills available without filling the context window.

Keep videos like this one searchable

BrainTube saves YouTube videos with their transcripts into one compiled knowledge base that Claude, ChatGPT, Cursor and other MCP clients can search.

Start free

The Four Types of Memory Every AI Agent Needs

In short

The argument

Key ideas, with timestamps

Starting from human memory

The CoALA framework

Working memory is the context window

Semantic memory is often a Markdown file

Procedural memory as skills

Episodic memory as distilled experience

Forgetting is the hard part

Match memory to the agent