Agent Memory

Why Agents Need Memory

LLMs are inherently stateless—each API call is independent with no memory of previous interactions. For agents to maintain context across conversations, learn from past experiences, and build relationships with users, they need explicit memory systems.

Memory enables agents to: remember user preferences, track task progress across sessions, learn from past mistakes, and provide personalized, contextual responses.

Types of Agent Memory

Working Memory (Context Window)

The simplest form of memory is the LLM's context window itself. Messages are accumulated and passed to the model with each request.

# Simple conversation memory
from openai import OpenAI
client = OpenAI()

messages = [
    {"role": "system", "content": "You are a helpful assistant."}
]

def chat(user_message):
    messages.append({"role": "user", "content": user_message})
    
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=messages  # Full history passed each time
    )
    
    assistant_message = response.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_message})
    
    return assistant_message

# Conversation
chat("My name is Alex")
chat("What's my name?")  # → "Your name is Alex"
            

Summarization Memory

Instead of keeping full history, periodically summarize older messages to compress context while preserving key information.

# Summarization memory pattern
from langchain.memory import ConversationSummaryMemory
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o")

memory = ConversationSummaryMemory(
    llm=llm,
    return_messages=True
)

# After many exchanges, memory.buffer contains:
# "The user Alex discussed Python best practices, asked about
#  async programming, and requested help with FastAPI routing."
            

Vector-Based Memory (Episodic)

Store conversation chunks or summaries as embeddings. Retrieve relevant memories based on semantic similarity to the current context.

from langchain.memory import VectorStoreRetrieverMemory
from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma

# Initialize vector store for memories
embeddings = OpenAIEmbeddings()
vectorstore = Chroma(
    collection_name="agent_memory",
    embedding_function=embeddings,
    persist_directory="./memory_db"
)

memory = VectorStoreRetrieverMemory(
    retriever=vectorstore.as_retriever(search_kwargs={"k": 5})
)

# Save a memory
memory.save_context(
    {"input": "I'm working on a machine learning project"},
    {"output": "That sounds interesting! What kind of ML are you focusing on?"}
)

# Later: retrieve relevant memories
relevant = memory.load_memory_variables(
    {"input": "Help me with my ML project"}
)
# → Returns the earlier conversation about ML
            

Structured Memory (Semantic)

Store explicit facts and user preferences in a structured format (JSON, database, knowledge graph) for reliable retrieval.

# Structured user profile memory
import json

class UserMemory:
    def __init__(self, user_id):
        self.user_id = user_id
        self.profile = self._load_or_create()
    
    def _load_or_create(self):
        try:
            with open(f"memories/{self.user_id}.json") as f:
                return json.load(f)
        except FileNotFoundError:
            return {
                "name": None,
                "preferences": {},
                "facts": [],
                "past_tasks": []
            }
    
    def remember(self, key, value):
        """Store a fact or preference"""
        if key == "name":
            self.profile["name"] = value
        elif key.startswith("prefers_"):
            self.profile["preferences"][key] = value
        else:
            self.profile["facts"].append({"key": key, "value": value})
        self._save()
    
    def recall(self, key):
        """Retrieve a specific memory"""
        if key == "name":
            return self.profile["name"]
        return self.profile["preferences"].get(key)
    
    def get_context(self):
        """Generate context string for LLM"""
        ctx = []
        if self.profile["name"]:
            ctx.append(f"User's name: {self.profile['name']}")
        for k, v in self.profile["preferences"].items():
            ctx.append(f"User {k.replace('_', ' ')}: {v}")
        return "\n".join(ctx)
            

Persistent Memory with LangGraph

LangGraph provides built-in checkpointing for persistent agent state across sessions.

from langgraph.graph import StateGraph
from langgraph.checkpoint.sqlite import SqliteSaver

# Define agent state
class AgentState(TypedDict):
    messages: list
    user_preferences: dict
    task_history: list

# Create persistent checkpointer
checkpointer = SqliteSaver.from_conn_string("./agent_memory.db")

# Build graph with checkpointing
graph = StateGraph(AgentState)
# ... add nodes and edges ...
app = graph.compile(checkpointer=checkpointer)

# Each thread_id gets its own persistent memory
config = {"configurable": {"thread_id": "user_123"}}

# State automatically persists and resumes
result = app.invoke({"messages": [user_message]}, config)
            

Production Memory Architecture

Best Practices

• Layer your memory: Combine buffer + summary + vector for different time horizons
• Scope memories: Separate user-specific, session-specific, and global memories
• Prune regularly: Remove outdated or low-value memories to reduce noise
• Include metadata: Timestamp, topic, importance score for better retrieval
• Handle conflicts: When memories contradict, prefer more recent ones
• Privacy first: Allow users to view and delete their memories
• Test retrieval: Verify relevant memories are actually retrieved for queries

Related Topics

• LangGraph
• Embedding Models
• Retrieval-Augmented Generation
• Model Context Protocol (MCP)
• Agent Observability