Short-term vs long-term memory in AI agents

The two terms are borrowed from psychology and map neatly onto how agents are built.

Short-term memory

Short-term, or working, memory is the context window: the messages so far, tool outputs, the scratchpad. It is word-for-word accurate and immediately available. It is also limited in size and wiped when the session ends.

An agent manages it by trimming (dropping the oldest turns), summarising (replacing old turns with a shorter account) or both.

Long-term memory

Long-term memory is a store outside the model: a database of short notes, each tied to a user or project. It is not word-for-word. It holds what was judged worth keeping, and it is reached only through recall.

How they compare

	Short-term	Long-term
Lives in	The context window	An external store
Lasts	This session	Until changed or deleted
Holds	Exact messages	Extracted facts
Reached by	Being in the prompt already	Search, then insertion into the prompt
Fails by	Overflowing	Going stale

The handover

The interesting design question is when facts cross from one to the other. There are three good moments.

1. At the end of a session. One extraction pass over the whole conversation. Cheap, and the model sees the full story before deciding what mattered.
2. Before trimming or summarising. If older turns are about to leave the window, extract from them first. Otherwise a preference stated at turn 3 is lost by turn 60.
3. On an explicit signal. “Remember that I…” or “From now on…” should be written immediately and confirmed to the user.

Writing after every single turn is possible and costs a model call each time. Most products do not need it.

A worked example

A user plans a trip over forty messages. Short-term memory holds the draft itinerary and the back-and-forth. At the end, long-term memory receives three notes: prefers morning flights, travels with a child, has a budget ceiling. Next month, a new trip starts with those three facts already in the prompt and none of the forty messages.