AI Agent Memory: Short-Term, Long-Term, and Working Memory for LLM Agents
TL;DR — AI agent memory: three-tier architecture. RapidClaw: "Three tiers: short-term (context window), working memory (task state), long-term (persistent storage). Vector stores for semantic recall, databases for structured state. Chunk 200-500 tokens, top-K 3-10. Sliding window with summarization and retrieval." ACL 2026 AgeMem: "Unified LTM and STM management in agent policy. Memory as tool-based actions: store, retrieve, update, summarize, discard. Three-stage progressive RL with step-wise GRPO. Outperforms baselines on long-horizon tasks." AdMem arXiv: "Semantic, episodic, procedural memory in bi-level design. Multi-agent: actor, memory, critic. Reward-based evaluation, merging, pruning for scalability." Memoria arXiv: "Dynamic session-level summarization and weighted knowledge graph. Hybrid: short-term coherence + long-term personalization within token constraints." arXiv Memory Framework: "Four retrieval paradigms: lexical (BM25), vector (cosine similarity, HNSW), structure (graph traversal), LLM-assisted." Learn more with what is AI agent, build agent, tool loop, and multi-agent.
ACL 2026 AgeMem frames the core challenge: "Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows, making effective memory management critical. Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components, relying on heuristics or auxiliary controllers, which limits adaptability and end-to-end optimization."
RapidClaw describes the practical architecture: "AI agents need three tiers of memory: short-term (context window), working memory (current task state), and long-term (persistent storage across sessions). The context window is fast but finite. Long-term memory requires external infrastructure — vector stores for semantic recall, databases for structured state, and careful retrieval strategies to avoid stuffing irrelevant context."
Memory Architecture
reasoning, planning
tool selection"] end subgraph STM["Tier 1: Short-Term Memory"] Context["Context Window
current conversation
system prompt + messages
fast but finite
128K-200K tokens"] Sliding["Sliding Window
keep last N messages
discard older ones"] Summary["Summarization
compress old messages
into compact summary"] end subgraph Working["Tier 2: Working Memory"] TaskState["Task State
steps completed
intermediate results
next action"] DataModel["Structured Data Model
not bag of tokens
agent reads and writes
explicit schema"] end subgraph LTM["Tier 3: Long-Term Memory"] Semantic["Semantic Memory
facts, preferences
entity relationships
knowledge graph / database"] Episodic["Episodic Memory
past events
conversation history
vector store + timestamps"] Procedural["Procedural Memory
task patterns
reusable workflows
effectiveness scores"] end subgraph Retrieval["Retrieval Strategies"] Lexical["Lexical
BM25
exact term matching"] Vector["Vector
embedding similarity
cosine distance
HNSW / ANN search"] Structure["Structure
graph traversal
multi-hop reasoning
Mem0, Zep"] LLMAssist["LLM-Assisted
LLM as reasoning
component for retrieval
query transformation"] end subgraph Storage["Storage Infrastructure"] VectorDB["Vector Store
Pinecone, Weaviate
pgvector, Chroma"] Database["Database
PostgreSQL, Redis
Key-value, relational"] GraphDB["Knowledge Graph
Neo4j, Mem0
entity relationships"] end LLM --> Context Context --> Sliding Context --> Summary LLM --> TaskState TaskState --> DataModel LLM --> Semantic LLM --> Episodic LLM --> Procedural Semantic --> Retrieval Episodic --> Retrieval Procedural --> Retrieval Retrieval --> Lexical Retrieval --> Vector Retrieval --> Structure Retrieval --> LLMAssist Vector --> VectorDB Semantic --> Database Semantic --> GraphDB Episodic --> VectorDB Procedural --> Database style STM fill:#4169E1,color:#fff style Working fill:#39FF14,color:#000 style LTM fill:#2D1B69,color:#fff
Memory Types Comparison
| Type | Scope | Storage | Retrieval | Persistence |
|---|---|---|---|---|
| Short-term | Current conversation | Context window | Direct (in prompt) | Session only |
| Working | Current task | Structured data model | Direct (read/write) | Task duration |
| Semantic | Facts, preferences | Database, knowledge graph | Lexical, vector, structure | Permanent |
| Episodic | Past events | Vector store + timestamps | Vector similarity | Permanent |
| Procedural | Task patterns | Database + effectiveness | Effectiveness + similarity | Permanent (pruned) |
Retrieval Paradigms
| Paradigm | Mechanism | Best For | Examples |
|---|---|---|---|
| Lexical | Token overlap, BM25 | Exact term matching | Names, entities, phrases |
| Vector | Embedding similarity | Semantic recall | Past conversations, documents |
| Structure | Graph traversal | Multi-hop reasoning | Entity relationships |
| LLM-Assisted | LLM reasoning | Ambiguous queries | Query transformation |
Implementation
from dataclasses import dataclass
from typing import Optional
from enum import Enum
class MemoryType(Enum):
SHORT_TERM = "short_term"
WORKING = "working"
LONG_TERM_SEMANTIC = "semantic"
LONG_TERM_EPISODIC = "episodic"
LONG_TERM_PROCEDURAL = "procedural"
@dataclass
class AIAgentMemoryGuide:
"""AI agent memory implementation guide."""
def get_three_tier_memory(self) -> str:
"""Three-tier memory implementation."""
return (
"import json\n"
"from openai import OpenAI\n"
"\n"
"client = OpenAI()\n"
"\n"
"class AgentMemory:\n"
" '''Three-tier memory\n"
" for AI agents.'''\n"
" \n"
" def __init__(self):\n"
" # Tier 1: Short-term\n"
" self.messages = []\n"
" self.max_context = 50\n"
" \n"
" # Tier 2: Working memory\n"
" self.task_state = {\n"
" 'steps_completed': [],\n"
" 'intermediate_results': {},\n"
" 'next_action': None,\n"
" 'goal': None\n"
" }\n"
" \n"
" # Tier 3: Long-term\n"
" self.long_term = []\n"
" self.semantic_facts = {}\n"
" self.episodic_events = []\n"
" self.procedural_patterns = []\n"
" \n"
" # Tier 1: Short-term\n"
" def add_message(\n"
" self, role, content\n"
" ):\n"
" '''Add to context.'''\n"
" self.messages.append({\n"
" 'role': role,\n"
" 'content': content\n"
" })\n"
" # Sliding window\n"
" if len(self.messages) > \\\n"
" self.max_context:\n"
" self.summarize_old()\n"
" \n"
" def summarize_old(self):\n"
" '''Summarize older\n"
" messages.'''\n"
" old = self.messages[\n"
" :-10] # keep last 10\n"
" recent = self.messages[-10:]\n"
" \n"
" summary = client.chat\\\n"
" .completions.create(\n"
" model='gpt-4o',\n"
" messages=[{\n"
" 'role': 'system',\n"
" 'content': 'Summarize'\n"
" ' this conversation'\n"
" ' concisely.'\n"
" }, {\n"
" 'role': 'user',\n"
" 'content': json\\\n"
" .dumps(old)\n"
" }]).choices[0]\\\n"
" .message.content\n"
" \n"
" self.messages = [\n"
" {'role': 'system',\n"
" 'content':\n"
" f'Previous summary:'\n"
" f' {summary}'},\n"
" *recent\n"
" ]\n"
" \n"
" # Tier 2: Working memory\n"
" def update_task_state(\n"
" self, key, value\n"
" ):\n"
" '''Update working\n"
" memory.'''\n"
" self.task_state[key] = \\\n"
" value\n"
" \n"
" def get_task_state(self):\n"
" '''Get current task\n"
" state.'''\n"
" return self.task_state\n"
" \n"
" # Tier 3: Long-term\n"
" def store_fact(\n"
" self, key, value\n"
" ):\n"
" '''Store semantic\n"
" memory.'''\n"
" self.semantic_facts[\n"
" key] = value\n"
" \n"
" def store_event(\n"
" self, event\n"
" ):\n"
" '''Store episodic\n"
" memory.'''\n"
" self.episodic_events\\\n"
" .append(event)\n"
" \n"
" def store_pattern(\n"
" self, pattern,\n"
" effectiveness=0\n"
" ):\n"
" '''Store procedural\n"
" memory.'''\n"
" self.procedural_patterns\\\n"
" .append({\n"
" 'pattern': pattern,\n"
" 'effectiveness':\n"
" effectiveness,\n"
" 'frequency': 0\n"
" })\n"
" \n"
" def retrieve_relevant(\n"
" self, query, top_k=5\n"
" ):\n"
" '''Retrieve relevant\n"
" long-term memories.'''\n"
" # Vector similarity\n"
" # search would go here\n"
" # Using simple keyword\n"
" # match for demo\n"
" results = []\n"
" for event in \\\n"
" self.episodic_events:\n"
" if any(word in \\\n"
" str(event) \n"
" for word in \\\n"
" query.split()):\n"
" results.append(\n"
" event)\n"
" return results[:top_k]\n"
" \n"
" def prune_procedural(\n"
" self, epsilon=0.1\n"
" ):\n"
" '''Prune low-frequency\n"
" procedural memories.'''\n"
" self.procedural_patterns \\\n"
" = [p for p in \\\n"
" self.procedural_patterns\n"
" if p['frequency']\n"
" >= epsilon]"
)
def get_vector_store_memory(self) -> str:
"""Vector store for long-term memory."""
return (
"import openai\n"
"import numpy as np\n"
"\n"
"class VectorMemory:\n"
" '''Vector store for\n"
" long-term agent memory.'''\n"
" \n"
" def __init__(self):\n"
" self.embeddings = []\n"
" self.texts = []\n"
" self.metadata = []\n"
" \n"
" def store(self, text, meta):\n"
" '''Store memory with\n"
" embedding.'''\n"
" # Chunk: 200-500 tokens\n"
" chunks = self.chunk(\n"
" text, max_tokens=300)\n"
" for chunk in chunks:\n"
" emb = openai\\\n"
" .embeddings.create(\n"
" model=\n"
" 'text-embedding'\n"
" '-3-small',\n"
" input=chunk\n"
" ).data[0].embedding\n"
" self.embeddings\\\n"
" .append(emb)\n"
" self.texts.append(chunk)\n"
" self.metadata\\\n"
" .append(meta)\n"
" \n"
" def retrieve(\n"
" self, query, top_k=5\n"
" ):\n"
" '''Retrieve by semantic\n"
" similarity.'''\n"
" query_emb = openai\\\n"
" .embeddings.create(\n"
" model=\n"
" 'text-embedding'\n"
" '-3-small',\n"
" input=query\n"
" ).data[0].embedding\n"
" \n"
" # Cosine similarity\n"
" scores = []\n"
" for i, emb in enumerate(\n"
" self.embeddings):\n"
" score = np.dot(\n"
" query_emb, emb) / (\n"
" np.linalg.norm(\n"
" query_emb) * \n"
" np.linalg.norm(\n"
" emb))\n"
" scores.append(\n"
" (score, i))\n"
" \n"
" # Top-K results\n"
" scores.sort(\n"
" reverse=True)\n"
" return [\n"
" {'text': self.texts[i],\n"
" 'metadata':\n"
" self.metadata[i],\n"
" 'score': s}\n"
" for s, i in \n"
" scores[:top_k]\n"
" ]\n"
" \n"
" def chunk(\n"
" self, text, max_tokens\n"
" ):\n"
" '''Split into chunks.'''\n"
" words = text.split()\n"
" chunks = []\n"
" current = []\n"
" for w in words:\n"
" current.append(w)\n"
" if len(current) >= \\\n"
" max_tokens:\n"
" chunks.append(\n"
" ' '.join(\n"
" current))\n"
" current = []\n"
" if current:\n"
" chunks.append(\n"
" ' '.join(\n"
" current))\n"
" return chunks"
)
def get_memory_operations(self) -> dict:
"""AgeMem memory operations as tools."""
return {
"store": (
"Agent decides what to "
"store in long-term "
"memory. AgeMem: 'Memory "
"operations as tool-based "
"actions — store.'"
),
"retrieve": (
"Agent retrieves relevant "
"memories from long-term "
"storage. Vector similarity "
"or graph traversal."
),
"update": (
"Agent updates existing "
"memories with new "
"information. Semantic: "
"LLM manages information "
"updates."
),
"summarize": (
"Agent compresses older "
"context into summary. "
"Memoria: 'Dynamic "
"session-level "
"summarization.'"
),
"discard": (
"Agent discards irrelevant "
"or outdated memories. "
"AdMem: 'Prunes and evicts "
"redundant memories.'"
),
}
def get_context_strategies(self) -> dict:
"""Context window management."""
return {
"sliding_window": (
"Keep system prompt + "
"last N messages. Discard "
"older. Simple but loses "
"early context."
),
"summarization": (
"Compress older messages "
"into summary. Replace "
"old messages with "
"compact form. Preserves "
"key information."
),
"retrieval": (
"Store full history "
"externally. Retrieve "
"relevant chunks on "
"demand. Most flexible "
"but requires "
"infrastructure."
),
"hybrid": (
"Combine sliding window + "
"summarization + retrieval. "
"Best for long conversations. "
"RapidClaw: 'Context "
"management handles "
"summarization and "
"retrieval automatically.'"
),
}
def get_best_practices(self) -> dict:
"""Best practices."""
return {
"chunk_size": (
"200-500 tokens per chunk. "
"Too large: irrelevant "
"context. Too small: lose "
"surrounding meaning."
),
"top_k": (
"3-10 results. More = more "
"context but more cost "
"and noise. Tune per task."
),
"embedding_model": (
"Use consistent embedding "
"model for storage and "
"retrieval. text-embedding-"
"3-small is cost-effective."
),
"persistence": (
"Persist conversations with "
"configurable retention "
"(30, 90, 365 days). "
"RapidClaw pattern."
),
"pruning": (
"Prune low-frequency "
"procedural memories. "
"Merge co-occurring "
"memories. AdMem: "
"threshold epsilon for "
"disuse."
),
}
AI Agent Memory Checklist
- [ ] Three tiers of memory: short-term (context window), working memory (task state), long-term (persistent storage)
- [ ] Short-term: context window — current conversation, system prompt, messages. Fast but finite (128K-200K tokens)
- [ ] Working memory: structured task state — steps completed, intermediate results, next action. Not bag of tokens but data model
- [ ] Long-term: persistent across sessions — semantic (facts), episodic (past events), procedural (task patterns)
- [ ] Semantic memory: facts, user preferences, entity relationships — stored in database or knowledge graph
- [ ] Episodic memory: past events, conversation history — stored in vector store with timestamps
- [ ] Procedural memory: task patterns, reusable workflows — stored with effectiveness scores, pruned by frequency
- [ ] LLM agents face fundamental limitations in long-horizon reasoning due to finite context windows
- [ ] AgeMem: unified framework integrating LTM and STM management directly into agent policy
- [ ] AgeMem: memory operations as tool-based actions — store, retrieve, update, summarize, discard
- [ ] AgeMem: agent autonomously decides what and when to store, retrieve, update, summarize, or discard
- [ ] AgeMem: three-stage progressive reinforcement learning with step-wise GRPO for sparse rewards
- [ ] AgeMem: outperforms baselines on long-horizon tasks with improved context usage and memory quality
- [ ] AdMem: unified framework integrating semantic, episodic, and procedural memory in bi-level design
- [ ] AdMem: multi-agent architecture with actor, memory, and critic agents for automatic memory generation
- [ ] AdMem: reward-based evaluation, merging, and pruning for scalability and continual improvement
- [ ] AdMem: dense retrieval for episodic and semantic, effectiveness + similarity for procedural
- [ ] AdMem: prunes low-frequency memories (below epsilon threshold), merges co-occurring memories
- [ ] Memoria: dynamic session-level summarization and weighted knowledge graph for user modelling
- [ ] Memoria: hybrid architecture for short-term dialogue coherence and long-term personalization
- [ ] Vector stores: default solution for LLM memory — find relevant context by meaning, not keywords
- [ ] Vector stores: embed chunks, similarity search with cosine distance, ANN with HNSW
- [ ] Chunk size: 200-500 tokens per chunk — too large = irrelevant, too small = lose meaning
- [ ] Top-K retrieval: 3-10 results — more = more context but more cost and noise
- [ ] Insert retrieved chunks into prompt as relevant context section between system prompt and user message
- [ ] This is the core of retrieval-augmented generation (RAG)
- [ ] Four retrieval paradigms: lexical (BM25), vector (cosine similarity, HNSW), structure (graph traversal), LLM-assisted
- [ ] Lexical: token overlap, BM25 — exact term matching for names, entities, phrases
- [ ] Vector: embedding similarity — semantic recall for past conversations, documents
- [ ] Structure: graph traversal, multi-hop reasoning — Mem0, Zep with BFS-based traversal
- [ ] LLM-assisted: LLM as reasoning component for retrieval — query transformation, entity identification
- [ ] Context management: sliding window (keep last N, discard older)
- [ ] Context management: summarization (compress old messages into summary)
- [ ] Context management: retrieval (store externally, retrieve on demand)
- [ ] Context management: hybrid (sliding window + summarization + retrieval)
- [ ] Storage infrastructure: vector stores (Pinecone, Weaviate, pgvector, Chroma)
- [ ] Storage infrastructure: databases (PostgreSQL, Redis, Key-value, relational)
- [ ] Storage infrastructure: knowledge graphs (Neo4j, Mem0, entity relationships)
- [ ] Persistence: configurable retention (30, 90, 365 days) — RapidClaw pattern
- [ ] Memory operations as tools: store, retrieve, update, summarize, discard — AgeMem pattern
- [ ] Pruning: remove low-frequency procedural memories, merge co-occurring memories
- [ ] Read what is AI agent for architecture
- [ ] Read build agent for implementation
- [ ] Read tool loop for ReAct pattern
- [ ] Read multi-agent for coordination
- [ ] Test: short-term memory persists across loop iterations
- [ ] Test: working memory tracks task state correctly
- [ ] Test: long-term memory stores and retrieves facts
- [ ] Test: summarization compresses old messages without losing key info
- [ ] Test: vector retrieval returns relevant context by semantic similarity
- [ ] Test: pruning removes low-frequency memories
- [ ] Test: sliding window maintains context within token limit
- [ ] Document memory tiers, storage backends, retrieval strategy, chunk size, top_k, retention policy
FAQ
What are the types of AI agent memory?
AI agents use three tiers of memory: short-term (context window), working memory (current task state), and long-term (persistent storage). RapidClaw: "AI agents need three tiers of memory: short-term (context window), working memory (current task state), and long-term (persistent storage across sessions). The context window is fast but finite. Long-term memory requires external infrastructure — vector stores for semantic recall, databases for structured state." ACL 2026 AgeMem: "LLM agents face fundamental limitations in long-horizon reasoning due to finite context windows, making effective memory management critical. Existing methods handle long-term memory (LTM) and short-term memory (STM) as separate components, relying on heuristics. AgeMem integrates LTM and STM management directly into the agent policy. Memory operations exposed as tool-based actions — store, retrieve, update, summarize, discard." AdMem arXiv: "Unified memory framework integrating semantic, episodic, and procedural memory in bi-level design combining short-term and long-term stores. Multi-agent architecture with actor, memory, and critic agents enables automatic memory generation, reward annotation, and adaptive retrieval." Types: (1) Short-term: context window, current conversation, fast but finite. (2) Working: structured task state — steps completed, intermediate results, next action. (3) Long-term: persistent across sessions — semantic (facts), episodic (past events), procedural (task patterns).
How does long-term memory work for AI agents?
Long-term memory persists across sessions using vector stores for semantic retrieval and databases for structured state. RapidClaw: "Long-term memory requires external infrastructure — vector stores for semantic recall, databases for structured state, and careful retrieval strategies. Vector stores solve: how do you find relevant context when you do not know the exact query? Vector similarity search works on meaning. Chunk size: 200-500 tokens. Top-K retrieval: 3-10 results. Insert retrieved chunks into prompt as relevant context section. This is the core of RAG." arXiv Memory Framework: "Four retrieval paradigms: lexical-based (BM25), vector-based (embedding similarity, ANN search with HNSW), structure-based (graph traversal, multi-hop reasoning), LLM-assisted (LLM as reasoning component for retrieval). Mem0 explores relationships from nodes. Zep uses BFS-based graph traversal." Memoria arXiv: "Memoria: dynamic session-level summarization and weighted knowledge graph-based user modelling. Hybrid architecture enables short-term dialogue coherence and long-term personalization within token constraints." Long-term: (1) Vector stores: embed chunks, similarity search for relevant context. (2) Structured storage: relational/Key-value for facts with known schema. (3) Knowledge graphs: entity relationships, multi-hop reasoning. (4) Retrieval: lexical (BM25), vector (cosine similarity), structure (graph traversal), LLM-assisted.
What is the AgeMem unified memory framework?
AgeMem integrates long-term and short-term memory management directly into the agent policy as tool-based actions. ACL 2026 AgeMem: "AgeMem: unified framework that integrates LTM and STM management directly into the agent policy. Memory operations exposed as tool-based actions, enabling the LLM agent to autonomously decide what and when to store, retrieve, update, summarize, or discard information. Three-stage progressive reinforcement learning strategy with step-wise GRPO to address sparse and discontinuous rewards. Experiments on five long-horizon benchmarks show AgeMem consistently outperforms strong memory-augmented baselines across multiple LLM backbones, achieving improved task performance, higher-quality long-term memory, and more efficient context usage." AgeMem: (1) Memory as tools: store, retrieve, update, summarize, discard. (2) Agent autonomously decides memory operations. (3) Three-stage progressive RL training. (4) Step-wise GRPO for sparse rewards. (5) Outperforms baselines on long-horizon tasks. (6) Better context usage and memory quality.
How do you manage context window limitations in AI agents?
Use summarization, sliding window, and retrieval to manage finite context windows. RapidClaw: "Sliding window: every conversation is automatically persisted with configurable retention (30, 90, 365 days). Context management handles summarization and retrieval automatically — agents recall relevant past interactions without manual RAG pipeline setup." ACL 2026 AgeMem: "LLM agents face fundamental limitations in long-horizon reasoning due to finite context windows, making effective memory management critical." Memoria arXiv: "Dynamic session-level summarization enables short-term dialogue coherence within token constraints of modern LLMs." Strategies: (1) Sliding window: keep system prompt + last N messages, discard older. (2) Summarization: compress older messages into summary, replace with compact form. (3) Retrieval: store full history externally, retrieve relevant chunks on demand. (4) Memory operations: store important facts, discard irrelevant. (5) Working memory: structured state outside context window. (6) Hybrid: sliding window + summarization + retrieval for long conversations.
What are semantic, episodic, and procedural memory in AI agents?
Three types of long-term memory: semantic (facts), episodic (past events), and procedural (task patterns). AdMem arXiv: "Unified memory framework integrating semantic, episodic, and procedural memory in bi-level design. Long-term memory agent stores three types: semantic, episodic, and procedural memory used across turns and tasks. Memory retrieval: dense retrieval for episodic and semantic memory with similarity match. For procedural memory, incorporate effectiveness evaluation and context similarity. Memory management: agent prunes and evicts redundant memories. LLM manages semantic memories for information updates. Procedural memories: threshold for disuse of memories retrieved below frequency epsilon are deleted. Memories constantly retrieved or not together are merged." arXiv Memory Framework: "Structure-based retrieval exploits relational connections between memory entities. Mem0 explores relationships from nodes. Zep uses BFS-based graph traversal." Types: (1) Semantic: facts, user preferences, entity relationships — stored in knowledge graph or database. (2) Episodic: past events, conversation history — stored in vector store with timestamps. (3) Procedural: task patterns, reusable workflows — stored with effectiveness scores, pruned by frequency.
Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →