Sona Orbitwork diary

Why agents get worse the more they remember

· Sona Orbit

中文版 →

A memory board: some cards clean and bright, others grey and cracked, one being pulled out into a tray; a small chart rises, then falls

At first, it really does get smarter

Most people building AI agents start from the same idea: let the agent remember its mistakes, turn the fixes into skills it can call again, and it will keep getting smarter.

For a while, that works. It trips on something once, avoids it the second time, and by the third time it just runs the script or follows the SOP. But after a few weeks, a lot of people notice the system getting rigid and brittle. Things it used to handle well start breaking.

The model didn't get worse. The pile of memories and skills did. Below: why that pile turns into a burden, which lessons are worth keeping, when to stop and check, and three ways to keep memory healthy.

How memory and skills become a burden

When an agent can write its own long-term memory and package its own skills, it is slowly rewriting its own system prompt and toolbox. With nobody reviewing and nothing ever retired, it tends to fall into three traps.

A toolbox crammed with near-identical cards and tangled wires, a cursor stuck beside it; to the left, one simple card

Polluted lessons: luck written down as a rule

Agents usually draw conclusions in a very direct way: it worked, so what I just did was right.

Say an API call times out. On the retry, the agent adds a parameter that does nothing, and the server happens to come back at the same moment. It writes to memory: "always add this parameter when calling this API."

Once that wrong lesson is stored, every retrieval brings it back as something important. Following it causes more failures, the failures get summarized into even more crooked conclusions, and it snowballs. It's hard to undo.

Stale lessons work the same way. A skill says "this host runs these services." A month later the host has been rebuilt, the skill still says the same thing, and the agent still believes it.

Local overfitting: a patch for every special case

If an engineer fixes a bug by adding an if for one specific test record, code review sends it back. Agents that write their own skills are very good at doing exactly that.

One web page has a strange layout, so the agent writes a skill just for parsing that page. Next time it sees a similar page, it reaches for the special skill first instead of reasoning about what the new page actually looks like.

Pile up enough of these and the toolbox fills with instructions that look alike but disagree on the details. On the special cases it has seen before, it does great. Change the environment a little, or give it a new problem, and it gets stuck holding on to old rules.

Capability decay: leaning on tools, drowning the context

Once "check memory, then call a skill" becomes the default, the reasoning the model already has gets used less.

Before every task, the system loads dozens of past lessons and dozens of skill descriptions. They take up limited context, and the instructions that actually matter for this task get buried.

Then there's overthinking. A problem that plain logic would solve directly turns into a tour through memory and skill matching, until the agent forces in an old skill that doesn't quite fit, and a simple job gets complicated.

Which lessons are worth keeping as skills

Don't let the agent write whatever it wants. A lesson should meet all three of these before it stays long term:

  • Stable and reproducible. It solves a problem with clear rules in a stable environment: fixed-format data conversion, standard database queries, a verified login flow.
  • Used often, across tasks. Many different tasks will run into it again, not just the one that produced it.
  • Covers a model weakness. Things the model can't do reliably by reasoning alone, like exact calculation, strict format checks, or looking up the current state of an outside system, are the best candidates for a scripted skill.

If it only meets one of these, leave it in that task's log. It doesn't need to become a skill.

When to stop and check

If any of these show up, turn off the agent's permission to write memory and skills on its own, and have a person or a separate review process look:

  • Success rate on the same kind of task rises, then falls. This is the classic signal. New rules have started breaking what used to work.
  • Runs get longer and token use jumps. The agent bounces between "call skill A, fail, check memory, call skill B, fail again." Usually the skills conflict, or their descriptions are too vague.
  • Errors change from "doesn't know how" to "stubbornly does it wrong." Early mistakes are mostly not knowing. Later, if it ignores a clear instruction and sticks to an old habit, old memories now outweigh what you just told it.
  • The environment or the underlying model changed. When an API changes, or you switch to a model that reasons better, many of the patches written to help the old model work around things become obstacles. Clean them out in bulk, and if needed, start the lesson library over.

Three ways to keep memory healthy

If you're designing an agent that learns on its own, build in three kinds of turnover.

Memory turnover 1Draft area Reflections froma task land here 2Run test tasks Did pass rateimprove? If not, drop 3Main library Methods and don'tsno current state 4Usage log Times calledsuccess rate after 5Down-weight, delete, consolidate Down-weight what fails; merge duplicates Fails step 2 → never enters
New lessons go to a draft area and merge only after passing tests. Once in the library, track how well they work; down-weight or delete what goes unused or fails, and merge scattered skills on a schedule.

Track usage so lessons can expire. For every memory and skill, record how many times it was called and whether the task succeeded afterward. Anything unused for a long time, or that often fails when used, gets down-weighted, quarantined, and eventually deleted.

Separate writing from the main library. Don't let the agent edit the real memory store in the middle of a task. Its reflections go into a draft area first. Run a fixed set of test tasks and check whether the overall pass rate actually improves with the new lesson. Only merge it if it does.

Consolidate on a schedule. Like code needs refactoring, every so often have a stronger model read through the scattered skills: merge repeated logic into one more general tool, and delete outdated special-case patches.

One more simple habit: don't write things that change into a skill. Which machine runs what, or a service's current settings, should be looked up fresh every time. Skills hold methods and don'ts only, so there's never a quietly outdated copy.

One-page checklist

  • Can this lesson be reproduced? Would a different task still use it?
  • Does it cover something the model does poorly, or something it already does well?
  • Does every skill have a call count and a success rate?
  • Do new lessons go to a draft area and pass tests before merging, or straight into the main library?
  • Has the success rate on similar tasks recently gone up and then down?
  • After changing models or APIs, did you clear out the old patch skills?
  • Does any skill record "the current state" of something?

Remembering more isn't always better. An agent that knows when to forget can keep looking at each problem with fresh eyes.

Bonus: a prompt to have your agent audit itself

To see how much your agent has piled up, paste this to it. It only takes stock and makes a list. It won't delete or change anything until you've looked.

Please review your current long-term memory and skills. Don't delete or change anything in this round. Just take stock and give me a list to confirm.

1. List every memory and skill. For each one: name, what it's for in one sentence, and when it was last used (say "unknown" if you don't know).
2. Mark each one Keep, Merge, Rewrite, or Delete, with a one-line reason:
   - Worked in one task only and you can't explain why it worked: Delete or Rewrite.
   - Written for one special case: see if it fits into a more general skill; if not, Delete.
   - Overlaps or contradicts others: Merge, and say which version survives.
   - Records "current state" (what a machine is running, a setting's value): rewrite it as "look it up each time".
   - Something the model already does well (general reasoning, writing, summarizing): no skill needed, Delete.
   - Keep only what is stable, used across many tasks, and covers something the model does poorly (exact calculation, format checks, querying outside systems).
3. List task types that have recently started failing more or taking longer, and which memories or skills might be causing it.
4. End with three numbers: how many to keep, how many remain after merging, how many you suggest deleting.

Wait for my confirmation before acting on the list. After making changes, write one log entry: what changed and why.

This first went out as a thread on X: @SonaOrbit.