Append-only logs: immutable history with corrections, not deletes
An append-only log is a store where the only write operation is to add a new record to the end; existing records are never edited or removed. You correct by appending — a compensating or correction entry that supersedes an earlier one — so the mistake and its fix both survive as history. This gives you a complete audit trail of what was believed and when, and when the records are hash-chained the whole sequence becomes tamper-evident: you cannot quietly rewrite the past without breaking the chain. HiveMind's corpus is append-mostly for exactly this reason — corrections are new facts, not destructive edits.
The only operation is append
An append-only log is the simplest durable data structure that still does something useful: the only write is to add a record to the end. Records already in the log are never edited and never removed. There is no UPDATE, no DELETE — only APPEND and READ.
That constraint sounds limiting, and it is, deliberately. By giving up in-place mutation you gain three properties that mutable storage cannot offer at the same time: a total order over every change, a complete record of history, and — once you chain the records — tamper-evidence. The log stops being a place where state lives and becomes the authoritative sequence of everything that ever happened, with current state derived by replaying or folding the records. That is the core idea behind event-sourced agent memory: the events are the truth, and any view is a projection you can rebuild.
You correct by appending, not by overwriting
The interesting question is what you do when a record is wrong. In a mutable CRUD store you find the row and overwrite it. The new value replaces the old, and the old value is gone — there is no record that it ever existed, no record of when it changed, and no record of who or what changed it.
In an append-only log you cannot do that, so you do the honest thing instead: you append a correction. This is the same pattern accountants have used for centuries — you do not erase a wrong ledger entry, you post a compensating entry that offsets it, and both lines remain on the page. The original record stays exactly as it was written; a later record supersedes it. A reader resolving current state sees the correction and acts on it, but the wrong value is still there behind it, dated, as the answer the system used to believe.
This is the precise mechanism behind the companion piece on remembering errors: the memory keeps the mistake and the correction. The error is no longer a live claim — it has been superseded — but it remains visible as history, so the next agent that approaches the same situation inherits the warning rather than walking into the same wall. Deleting the wrong answer would have thrown away the only artifact that prevents the repeat.
Tamper-evidence comes from chaining
An append-only discipline by itself is a convention; nothing physically stops someone with write access from rewriting a record after the fact. You close that gap by hash-chaining: each record carries a hash of its contents plus the hash of the record before it. Change any byte of any historical record and its hash changes, which breaks the next record’s back-reference, which breaks the one after that — the damage cascades all the way to the head. You cannot quietly alter the past; you can only append to a verifiable present.
HiveMind builds this as a Merkle index rather than a flat chain, which buys the same tamper-evidence plus efficient comparison: two machines can diff their logs and sync only the records that differ, without shipping the whole history. The Merkle DAG and provenance deep-dive covers how the tree structure makes both the integrity check and the peer-to-peer delta sync cheap. Every machine holds a full local copy, and the data stays on your devices.
The contrast: mutable CRUD loses the story
A row in a mutable table answers one question: what is true now? That is often all an application needs. But for institutional memory it is the wrong shape, because the moment you overwrite a value you have answered “what is true now” by destroying “what was true before and when it changed.” There is no audit trail, no way to ask what the system believed last Tuesday, and no way to distinguish a fact that was corrected from one that was never recorded.
An append-only log answers a strictly larger set of questions: not just the current state, but the full sequence of beliefs and the exact transition between them. The cost is that the log grows, which is real — see the tombstones and compaction deep-dive for how superseded records are reclaimed without reintroducing destructive deletes. In HiveMind, forgetting is a deliberate soft decay, not a silent erase: even removal is an explicit, recorded act.
Why this is the right substrate for memory
The append-only log is not a clever optimization layered on top of storage; it is the storage model that matches what institutional memory actually needs. It keeps a total order so concurrent writers can be reconciled. It keeps the full history so nothing learned is lost. It keeps corrections beside the things they correct, so the system gets steadily harder to fool instead of quietly drifting. And because the chain is tamper-evident, the trail you rely on is one you can verify — not one you have to trust. Mutable storage optimizes for a clean present. An append-only log optimizes for a truthful past, which is the only foundation a memory you can learn from can stand on.
Frequently asked
If you never delete, doesn't the log grow without bound?
It grows monotonically, yes, but that is a storage problem, not a correctness problem, and it is bounded by compaction rather than by deletion-in-place. Superseded records can be folded away once no reader needs the intermediate states, and forgetting is modeled deliberately as decay rather than a silent erase. The point is that pruning is an explicit, auditable operation — not an in-place mutation that leaves no trace.
What's the difference between a write-ahead log and an append-only event log?
Mechanically they are the same primitive — both append durable records before anything else happens. The difference is intent. A write-ahead log (WAL) is an implementation detail of a database: it records the intended mutation, durably, so the system can recover or roll back, and it is usually truncated once the change is checkpointed into the main store. An append-only event log keeps those records as the system of record itself — the log is the truth, not a recovery aid for some other truth.
Why correct with a new entry instead of just fixing the wrong one?
Editing in place destroys the evidence that the system was ever wrong, which is the one record that stops the mistake from recurring. A compensating entry keeps both states: the original claim, marked superseded, and the correction, marked current. Readers act on the latest, but the trail of what was believed and when stays intact and auditable.
Related
Take yourself out of the loop.
Let your agents do the lifting while you keep the judgment.
Get the Playbook