Agent Memory Distillation

Small agents can't learn from their own failures. Give them the teacher's memory instead.

Small agents rarely succeed enough to build useful memory, and naive teacher-memory transfer fails on the capability gap. AMD closes it, training-free, with memory at three granularities: Workflow plans, Subtask examples, and Function call guides.

Figure 1 · Motivation & conceptCLICK TO ENLARGE

Three memories, two injection modes

Built only from successful teacher trajectories. Planning knowledge arrives before execution; corrective knowledge arrives at the moment of failure.

Figure 2 · Memory generation & injection pipelineCLICK TO ENLARGE

Students approach and even surpass their teacher

AMD beats every memory baseline on every model × benchmark, while baselines often fall below zero-shot. It also cuts interaction turns toward the teacher's efficiency.

Figure 3 · Interaction efficiency: steps needed to complete a taskCLICK TO ENLARGE

What makes distillation work

Five ablations answer: which memory matters most, which teacher is best, which student sizes benefit most, how many memories to retrieve, and how to represent them.

FIGURE 4 · MODEL SIZE

4B students benefit most

Absolute gain peaks at +34.5 %p for Qwen3-4B; larger models close the gap without help.

FIGURE 5 · RETRIEVAL COUNT

Retrieve one, not many

k = 1 is near-optimal. Subtask accuracy collapses 49.4 to 33.3 as k grows to 5. Quality beats quantity.

Each memory fixes a distinct failure

One Venmo task, three cascading failures. Workflow fixes the month-scope filter, Subtask escapes a date-parsing loop, Function fixes a dict-key crash. All three were necessary: ✓ solved in 7 steps.

Figure 6 · Full case study (WF → ST → FN)CLICK TO ENLARGE

BibTeX

@misc{kim2026agentmemorydistillationempowering,
      title={Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory},
      author={Taeil Kim and Kangsan Kim and Sung Ju Hwang},
      year={2026},
      eprint={2608.07169},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2608.07169},
}