AI Agent Memory: Why Your Agent Forgets After 50 Messages (And the Fix)

TL;DR: When a Pydantic AI agent runs for 50+ messages, early context silently disappears as the context window fills up — and tool-heavy agents hit this limit even faster, since a 20-turn conversation with 3 tool calls per turn consumes 80+ messages. The summarization-pydantic-ai library provides two drop-in processors: SummarizationProcessor, which uses an LLM to intelligently compress older messages into a summary while preserving tool call pairs, and SlidingWindowProcessor, which simply trims old messages at zero cost. Both support three trigger types — message count, token count, and fraction-based (e.g., 80% of context window) — with fraction-based being the most portable across models. The critical implementation detail both processors handle is tool call pair preservation: cutting between a tool call and its response creates orphaned calls that break the model, so the cutoff algorithm searches 5 messages in each direction to keep pairs intact.

Run a Pydantic AI agent for 50 messages and ask it about something from message #3. It won’t remember. Not because the model is bad - because the conversation got too long, the context window filled up, and the early messages fell off.

This isn’t a model problem. It’s an infrastructure problem. And it gets worse with agents that use tools, because every tool call and response is a message. A 20-turn conversation with 3 tool calls per turn is actually 80+ messages. At ~4 characters per token, that’s tens of thousands of tokens consumed before you know it.

TL;DR

• Context overflow is an infrastructure problem, not a model problem. Every long-running agent needs a context management strategy.
• SummarizationProcessor uses an LLM to intelligently compress old messages into a summary - higher quality, but costs an API call per trigger.
• SlidingWindowProcessor simply trims old messages for zero cost - fast but loses context entirely.
• Tool call pair preservation is critical. Both processors ensure tool calls and their responses are never split apart.
• Fraction-based triggers (("fraction", 0.8)) are the most portable option - they adapt automatically to any model’s context window.

We hit this wall in every long-running agent deployment. Customer support bots that forget the customer’s name. Code assistants that lose track of which files they’ve already modified. Research agents that re-investigate topics they covered 30 messages ago.

So we built summarization-pydantic-ai - two processors that manage your agent’s conversation history: one that summarizes intelligently using an LLM, and one that simply trims old messages for zero cost.

Two Processors, Two Trade-offs

Both work as Pydantic AI history processors - drop-in functions that transform the message history before each agent run.

SummarizationProcessor: Intelligent Compression

from pydantic_ai import Agent
from pydantic_ai_summarization import create_summarization_processor

processor = create_summarization_processor(
    model="openai:gpt-4.1",
    trigger=("tokens", 100000),  # Trigger at 100k tokens
    keep=("messages", 20),       # Keep last 20 messages
)

agent = Agent(
    "openai:gpt-4.1",
    history_processors=[processor],
)

When the token count exceeds the trigger, the processor:

1. Calculates a safe cutoff point (never splitting tool call pairs)
2. Sends the older messages to an LLM for summarization
3. Replaces the old messages with a compact summary
4. Keeps the last 20 messages intact

The result: your agent maintains context from early in the conversation without consuming the full token budget.

Three Trigger Types

You can trigger summarization based on messages, tokens, or context fraction:

Message-based - simple and predictable:

processor = SummarizationProcessor(
    model="openai:gpt-4.1",
    trigger=("messages", 50),  # After 50 messages
)

Token-based - accounts for message length:

processor = SummarizationProcessor(
    model="openai:gpt-4.1",
    trigger=("tokens", 100000),  # After 100k tokens
)

Fraction-based - adapts to any model’s context:

processor = SummarizationProcessor(
    model="openai:gpt-4.1",
    trigger=("fraction", 0.8),    # At 80% of context
    max_input_tokens=128000,       # GPT-4o's context window
)

Multiple triggers - OR logic, first one wins:

processor = SummarizationProcessor(
    model="openai:gpt-4.1",
    trigger=[
        ("messages", 50),     # OR
        ("tokens", 100000),   # OR
        ("fraction", 0.8),
    ],
    max_input_tokens=128000,
)

Fraction-based triggers are the most robust for production - they adapt automatically when you switch between models with different context windows. If you’re deciding which framework to build your agent with, our framework comparison covers how each one handles context management.

Tool Call Pair Preservation

This is the detail most context management solutions get wrong. Consider this message sequence:

User: "Search for Python tutorials"
Assistant: [tool_call: search("Python tutorials"), id=call_1]
Tool: [tool_return: "Found 5 results...", id=call_1]
Assistant: "Here are the top results..."
User: "Tell me more about the first one"

If you cut between the tool call and its return, the model sees an orphaned tool call - and breaks. Our processors handle this:

def _is_safe_cutoff_point(self, messages, cutoff_index):
    """Check if cutting at index would separate tool call/response pairs."""
    search_start = max(0, cutoff_index - 5)
    search_end = min(len(messages), cutoff_index + 5)

    for i in range(search_start, search_end):
        msg = messages[i]
        if not isinstance(msg, ModelResponse):
            continue

        tool_call_ids = set()
        for part in msg.parts:
            if isinstance(part, ToolCallPart) and part.tool_call_id:
                tool_call_ids.add(part.tool_call_id)

        if not tool_call_ids:
            continue

        # Check if cutoff separates this tool call from its response
        for j in range(i + 1, len(messages)):
            check_msg = messages[j]
            if isinstance(check_msg, ModelRequest):
                for part in check_msg.parts:
                    if (isinstance(part, ToolReturnPart)
                        and part.tool_call_id in tool_call_ids):
                        tool_before = i < cutoff_index
                        response_before = j < cutoff_index
                        if tool_before != response_before:
                            return False  # Unsafe - would split pair
    return True

The processor searches 5 messages in either direction from the cutoff point, finds all tool call/return pairs, and ensures they stay on the same side of the cut.

SlidingWindowProcessor: Zero-Cost Trimming

When you don’t need intelligent summarization - when speed and cost matter more than context quality:

from pydantic_ai_summarization import create_sliding_window_processor

processor = create_sliding_window_processor(
    trigger=("messages", 100),  # Trim at 100 messages
    keep=("messages", 50),      # Keep last 50
)

agent = Agent(
    "openai:gpt-4.1",
    history_processors=[processor],
)

No LLM call. No latency. Old messages are simply discarded. The same tool call pair preservation applies - the sliding window won’t cut in the middle of a tool call sequence.

Token Counting: Approximate by Default

The default token counter uses a simple heuristic - ~4 characters per token:

def count_tokens_approximately(messages):
    total_chars = 0
    for msg in messages:
        if isinstance(msg, ModelRequest):
            for part in msg.parts:
                if isinstance(part, UserPromptPart):
                    if isinstance(part.content, str):
                        total_chars += len(part.content)
                elif isinstance(part, SystemPromptPart):
                    total_chars += len(part.content)
                elif isinstance(part, ToolReturnPart):
                    total_chars += len(str(part.content))
        elif isinstance(msg, ModelResponse):
            for part in msg.parts:
                if isinstance(part, TextPart):
                    total_chars += len(part.content)
                elif isinstance(part, ToolCallPart):
                    total_chars += len(part.tool_name)
                    total_chars += len(str(part.args))
    return total_chars // 4

This is fast and good enough for most cases. For precision, plug in tiktoken:

import tiktoken

def accurate_counter(messages):
    encoding = tiktoken.encoding_for_model("gpt-4")
    total = 0
    for msg in messages:
        total += len(encoding.encode(str(msg)))
    return total

processor = create_summarization_processor(
    token_counter=accurate_counter,
    trigger=("tokens", 100000),
)

Custom Summary Prompts

The default summary prompt extracts key context. You can customize it:

processor = create_summarization_processor(
    summary_prompt="""
    You are summarizing an agent conversation. Extract:

    1. **Key Decisions**: What was decided?
    2. **Code Changes**: What code was written/modified?
    3. **Pending Tasks**: What still needs to be done?
    4. **Important Context**: What context is crucial to preserve?

    Conversation to summarize:
    {messages}

    Provide a concise summary that preserves essential information.
    """,
)

The {messages} placeholder is replaced with the formatted message history. Customize this for your domain - a customer support agent might extract customer name and issue, while a code assistant might focus on file changes and test results.

Conversation Loop Pattern

Here’s the typical production pattern:

from pydantic_ai import Agent
from pydantic_ai_summarization import create_summarization_processor

processor = create_summarization_processor(
    trigger=("messages", 20),
    keep=("messages", 5),
)

agent = Agent(
    "openai:gpt-4.1",
    history_processors=[processor],
)

async def chat():
    message_history = []

    while True:
        user_input = input("You: ")
        if user_input.lower() == "quit":
            break

        result = await agent.run(
            user_input,
            message_history=message_history,
        )

        print(f"Assistant: {result.output}")
        message_history = result.all_messages()

The history processor runs automatically before each agent.run() call. You pass the full message history, the processor checks if summarization is needed, and returns a (possibly compressed) history. No manual token counting required.

Key Takeaways

• Context overflow is an infrastructure problem, not a model problem. Every long-running agent needs a context management strategy. Choose between intelligent summarization (higher quality, LLM cost) and sliding window (zero cost, context loss). For a deeper dive into what agents should and shouldn’t remember, see why your AI agent remembers too much.
• Tool call pair preservation is critical. Never cut between a tool call and its response. Both our processors handle this automatically - search 5 messages in each direction for orphaned pairs.
• Fraction-based triggers are the most portable. ("fraction", 0.8) works regardless of which model you use or how long your messages are. Token and message counts are model- and content-specific.
• The ~4 chars/token heuristic is good enough. For production precision, use tiktoken. For everything else, the default counter keeps things fast without adding a dependency.
• Customize the summary prompt for your domain. The default extracts generic context. A domain-specific prompt (“extract customer name, issue ID, and resolution status”) produces much better summaries.

Try It Yourself

summarization-pydantic-ai - Automatic conversation summarization and context management for Pydantic AI agents.

pip install summarization-pydantic-ai