Event-sourced agent memory: deriving state from an append-only event stream

State is a lie you keep overwriting

The default way to store what an agent knows is a row you update in place: a key, a current value, and every write clobbers the last. It answers “what is true now?” in one lookup, which is why it is the default. But it answers nothing else. The moment you overwrite, you destroy the history of how you arrived at the current value — what was tried, what was corrected, why a decision was made. For agent memory that is the expensive part to lose, because the next agent re-derives exactly what the overwrite threw away.

Event sourcing inverts the relationship between facts and state. Instead of storing current state and treating the changes as transient, you store the changes — the events — as the durable source of truth, and treat current state as something you compute from them. The log of what happened is primary; any snapshot of “what we know now” is secondary and disposable, because you can always rebuild it.

The log is truth; state is a fold over it

An event is an immutable statement that something occurred: observed X, decided Y for reason Z, corrected an earlier claim. Events are only ever appended, never mutated — which is the same discipline behind an append-only log architecture. Current state is then a left-fold over the stream:

state = events.reduce(apply, initial)

apply is a pure function: given a prior state and the next event, it returns the new state. Run it over the whole history and you get the present. This is why state is never something you trust on its own — it is always reconstructable, and the events are the thing you actually keep.

Projections, read models, and CQRS

The fold above produces one view, but a stream can feed many. A projection (also called a materialized view or read model) is a derived structure shaped for a particular read pattern — a by-topic index, a confidence summary, a timeline. Each projection subscribes to the same event log and maintains its own answer to its own question.

That naturally splits the system in two: a write side that only ever appends events, and a read side made of projections optimized for queries. Separating the command path from the query path this way is CQRS (Command Query Responsibility Segregation). The write model stays dead simple — validate and append — while read models can be added, dropped, reshaped, or denormalized freely, because none of them is the source of truth. If a projection’s logic was wrong, you fix the code and rebuild it; the events are untouched.

Log replay: rebuild and re-interpret

Because the events are retained, log replay does two distinct jobs. The first is recovery: a corrupted or new read model is rebuilt by replaying events from offset zero. The second, and the one that matters more for memory, is re-interpretation. A question you did not think to ask when the events were written can be answered later by replaying the same history through a new projection. The past is not frozen into one interpretation — it is raw material you can fold differently as your understanding improves.

This is also why idempotent apply is non-negotiable. In a peer-to-peer system events can be delivered more than once or arrive out of order. If apply is idempotent — keyed on a stable event id or per-stream sequence number, with each projection tracking its high-water offset — then replay, re-sync, and duplicate delivery all converge to the same read model instead of double-counting. It is the same convergence discipline that CRDTs enforce at the data-structure level, applied here at the event level.

Snapshots: paying down the fold

Folding a long history on every read is wasteful. A snapshot is a cached projection state at a known offset: rehydrate from the snapshot, then replay only the events after it. The snapshot is purely an optimization — it is derived, never authoritative — so a stale or wrong snapshot is harmless; you discard it and replay from an earlier point. Fast rehydration without making the cache load-bearing.

How HiveMind applies it

The twin of this idea is that agent activity becomes memory: the observations, decisions, and corrections agents emit are not telemetry about the memory — they are the memory. HiveMind’s corpus is append-mostly, and corrections are new facts appended rather than edits to old ones, which is exactly the event-sourced contract — nothing is overwritten, so nothing is forgotten. Each machine holds a full local copy synced peer-to-peer, and your data stays on your devices, so every replica can fold the same stream into the same view.

Any answer to “what do we know now?” is a projection over that history, not a privileged record. And trust is a further fold: confidence is earned through corroboration, layered on top of the raw event stream rather than baked into it. The log is what happened; what to believe is a read model built from it.

Frequently asked

Related