engram.md

What is an engram?

An engram is a memory layer for a transformer: a hash-keyed lookup table over the last few tokens whose values are injected into the model's residual stream through a learned gate. Recall is a deterministic O(1) table lookup — no attention, no retrieval pipeline — so a model can carry vast static knowledge without spending reasoning compute on it.

The idea comes from DeepSeek's Conditional Memory via Scalable Lookup (reference code): where mixture-of-experts sparsifies computation, engrams sparsify memory access — a second axis of sparsity. It is already shipping at scale: Qwen3.8-Flash-Next, Alibaba's preview of the Qwen4 architecture, dedicates 51B of its 125B parameters to an engram n-gram table.

Detachable cartridges

Those built-in engram layers are baked in at pretraining time. This project takes the next step: engrams trained after the fact, as detachable cartridges for a frozen model — a way to securely embed your facts into your own model, on your own infrastructure.

• Mounts in milliseconds, unmounts to byte-identical. A cartridge is a sidecar file of a few MB. Remove it and the base model is provably unchanged — the knowledge is gone, not merely suppressed.
• Exact and auditable. Facts are stored against a canonical key contract (entity | attribute | → value). Every fact in a cartridge can be enumerated and verified against its source of truth.
• Continuously updated and pruned. Rebuilding a cartridge from the current source data takes minutes on a small model; swap it atomically. A pruned fact does not linger the way it does in fine-tuned weights.
• Sovereign, at two scales. Run it individually — a per-agent cartridge on a small local model, even entirely inside a browser tab, as the demos on this site do — or as a centrally served model in an organisation: one frozen base, per-team fact cartridges mounted per request. Your facts never leave your infrastructure or enter a third-party model.

In practice: LoRA adapters make good behaviour cartridges (format, style, tool habits); engrams make good fact cartridges. The demo stacks both on one 0.6B frozen base.

The engram.md file

Just as AGENTS.md gives coding agents a shared, human-readable operating manual, ENGRAMS.md is its machine-checkable counterpart for fact memory — an open format, specified at engramsmd/engrams.md. It lives in the agent's directory and declares the cartridges an agent carries:

## adapter

Premier League 2025-26

- **Key format**: `Tottenham v Man United | result | <value>`
- **Facts**: 380 · **Size**: 5.0 MB
- **Base model**: Qwen/Qwen3-0.6B-Base
- **Fingerprint**: `acfd09e8f97bb09e`

Key formats, fact counts, data sources, and the exact base-model fingerprint each cartridge is bound to. Fingerprints make the contract enforceable: a cartridge is trained against one exact stack and refuses to mis-mount on another. Agents can discover what they know, prove where it came from, and verify it before mounting — see the live ENGRAMS.md for the demo cartridges, and the public registry.

See it run

The in-browser demo loads a frozen Qwen3-0.6B base into this tab, then mounts and unmounts cartridges live: two Premier League seasons (380 results each — both seasons finished after the model's training cutoff, so the base provably cannot know them) and the populations of the world's 16,000 largest cities. Its Chat workspace shows the pattern an agent would use: ask "what is the population of Nairobi?" and a small instruct model decides what to look up, writes the memory's exact key, the mounted cartridge answers it, and the model phrases the result — with a provenance badge on every lookup. Unmount the memory and it forgets, and says so. Nothing you type leaves the tab.