Preprint: insights linked to source runs reduced harm from agent memory in tests
Written by AI
KAIST’s WikiTrace system lets an AI agent check an insight condensed into its memory against the run it came from. On five benchmarks, the agent using WikiTrace failed less often, on average, on tasks it solved without memory than it did with the same insights but no links or checking prompt.
Researchers from KAIST and DeepAuto.ai say in a preprint published on arXiv on 10 November 2026 that an AI agent’s memory can impair its performance. The work has been submitted to ACL Rolling Review for peer review, which is still under way, arXivDaily reported.
An agent’s memory usually holds insights condensed from earlier runs. A run is the sequence of the agent’s actions and observations. The only thing known about a run’s outcome is whether the task succeeded. The model that writes the insight therefore has to guess which action decided the outcome.
The researchers examined tasks that the agent always solved without memory but failed with it. A language-model judge attributed 52 per cent of these failures to a memory item the agent followed. Most often the item was wrong. The classification describes the content of the agent’s reasoning, not a definite cause.
From insight to source run
WikiTrace stores each insight with the unaltered source run and a summary of why the run succeeded or failed. The agent receives all insights at the start of a task. Before acting, the agent can read the linked files with command-line tools and correct an insight that is incomplete, conflicts with another or comes from a failed run.
Results on five benchmarks
The five benchmarks contained eight suites in total, including text-based household and science tasks and API use. All methods used the same agent and the DeepSeek V4 Flash model, and each test was run three times.
The table shows suite averages. The average is not a single metric, because the suites measure different things, such as success rate or score.
| Method | Average |
|---|---|
| No Memory | 79.5 |
| ExpeL | 80.3 |
| AWM | 81.6 |
| ReasoningBank | 81.7 |
| Insights Only | 87.0 |
| WikiTrace | 90.2 |
Source: Table 1 of the preprint. Figures are three-run averages across eight suites.
Regression is the share of tasks the agent solved without memory but failed with it. WikiTrace’s regression was 4.5 per cent. With Insights Only, the same insights without source runs, it was 6.5 per cent. When the agent was rerun without memory, 9.7 per cent of successes turned into failures through run-to-run variation alone.
The Insights Only comparison also lacked the prompt that directs the agent to check the sources. The advantage comes from both the source material and this prompt, the authors said.
On three text-based benchmarks, the agent opened the source run in 22 per cent of easy tasks and 44 per cent of hard ones. An easy task was one the agent solved without memory on all three attempts; a hard task was one it never solved.
In openings the agent made after getting stuck or doubting an insight, WikiTrace led Insights Only by 10.4 points, according to the language-model judge’s classification. In other openings the gap was 3.7 points. The agent decided for itself when to open a run, so the gaps do not show that opening caused them.
A link does not prove an insight correct
The link lets the agent check an insight but does not guarantee that the insight is correct, the authors said. The summary’s explanation of a failure may also be wrong. All insights go into the agent’s first message, so the prompt grows with the memory. The authors leave open how the agent would find the insights it needs in a larger memory.
The code is published on GitHub (github.com/yukyeongleee/wikitrace). Installation requires Python, Node, Docker and the uv tool, Blockchain.news reported.