How agents remember
Memory shapes the identity of an agent: it determines whether the system is a transient calculator, a collaborative colleague, or a permanent keeper of record. In melchizedek, this identity is defined by a single line of YAML through the memory_system parameter, offering three distinct tiers of retention.
internal-only is intentional transience. State lives solely within the active process and expires the moment it concludes. This setting fits test environments, batch jobs, and isolated pipelines, where state carried in from an earlier run is a source of contamination rather than help.
session-only persists the immediate conversation. Every interaction writes to Postgres, so a system restart resumes mid-thought. One structural detail protects this process: session rows are keyed by a composite of application, user, and session, which keeps parallel subagents under a global session from reading each other’s rows. The isolation holds only as long as every writer supplies all three parts of the key; the corrupted context that unravels multi-agent systems is exactly what the composite exists to prevent.
long-term lets facts about a user accumulate across sessions. When a session concludes, a model reads the transcript and distills it into structured records. Each record carries a tag such as [FACT], [PREFERENCE], or [CORRECTION], along with the date it is about, the source who asserted it, its exact values with units, its active or historical status, and the index keys it is filed under. These records are embedded as 768-dimension vectors and written to a pgvector table, scoped securely to that specific application and user, where the parsed fields live beside the vector as queryable columns.
recall is structural
Retrieval directly mirrors the storage process. When a new query arrives, it undergoes identical embedding, and a cosine-similarity search identifies the nearest relevant records. The structured fields then sharpen the ranking: a record gains standing when the query names one of its index keys or the month it is dated to, and an active record outranks one that a later correction has retired. These records merge into the agent’s context before it forms a response. Nothing here walks the history in order; the past surfaces when it resembles the present question, and stays where it is otherwise. Human memory works the same way: a certain smell can surface one summer from thirty years ago, not because you searched every year in order, but because the present moment resembled the past one.
The record also evolves. When a user corrects an earlier fact, the correction names the outdated claim, and the store retires the old record: marked superseded, linked to its replacement, preserved as history. A retired record that resurfaces in a search arrives clearly labeled, so an old value does not wear the authority of the current one. That protection reaches exactly as far as the corrections people actually speak: the store cannot retire a fact nobody amended.
The store also has to defend itself against its own diligence. A served agent is stateless between requests, so it distills after every completed task, and each pass once read the whole session: a twenty-turn conversation was distilled twenty times, and because the distilling model rephrases on every pass, a byte-for-byte comparison caught none of the repeats. One production store grew to 43 rows holding roughly seven distinct facts. Two mechanisms now keep ingestion honest. A per-session high-water mark means each pass distills only the turns added since the last one. And a semantic probe runs before every insert, dropping any candidate that lands above 0.93 similarity to a stored record carrying the same tag — the tag must match, so an [EPISODE] narrating a session can never suppress the [FACT] it mentions. The probe obeys one asymmetry: a duplicate costs a shortlist slot, while a lost fact costs the user something they said. So a probe that errors stores the record anyway, and a superseded record never suppresses a new one.
What deserves distilling is a domain judgment, and the extraction prompt is shared by every long-term agent. A syndicate therefore declares its own rules in a memory_extraction_rules field, appended to the shared prompt at run time: never store the values that go stale on their own, always store what the user asserted, decided, or committed to. An agent that declares none runs the shared prompt exactly as before.
Here is the whole lifecycle: a fact spoken, distilled, and recalled a session later.
Three costs come with this architecture, and all three are worth knowing before you build on it. Distillation is lossy, so whatever the extraction pass did not keep is gone with the session. The embedding dimension is a hard dependency: change the embedding model and the whole store has to be rebuilt. And a retrieved fact is context, not truth, so a fact recorded wrong stays wrong until somebody corrects it.
Memory turns a utilitarian tool into a genuine relationship: give a simple search interface a durable record and it becomes a patient advocate capable of recalling vital lab results at the precise moment they are needed. An agent that remembers a person is holding something that belongs to that person, which makes the store a responsibility rather than a feature.