---
title: "kmem architecture"
type: reference
tags: [signpost, docs, reference]
proposed_by: tok_bc09b340db294c709763789154d8113d
model: kmem-docs-publisher
extracted_at: "2026-10-08T22:25:37.346Z"
---
# kmem architecture

How the system is put together, described from the code in this repository rather than from a plan.
Every claim here is either a line you can read or a test you can run, and the test is named. This
document does not track state: `RELAUNCH_NOTES.md` at the repository root says what is broken now.

[`docs/ADR/`](./ADR) holds the decisions and the alternative each one rejected.
[`CODEBASE.md`](./CODEBASE.md) says where each thing lives.

---

## 1. What the system is, in one paragraph

kmem stores a team's knowledge as ordinary Markdown files in R2, derives a searchable graph of
them into D1 and Vectorize, and serves that graph to people in the web app and to AI agents over MCP
and over a pasteable URL. The Markdown is the product. Everything in D1 and Vectorize is a rebuildable
index of it, and the system is designed so that losing the index costs a reindex and losing the bucket
costs the company.

Three rules follow from that, and everything below is an elaboration of them:

1. **Markdown is ground truth.** R2 plus a git clone is the whole knowledge base.
2. **Retrieval is the product.** `recall` returns the right paragraphs inside a token budget, and
   that is what kmem is judged on.
3. **One write path.** Every mutation goes through `CommitService.commit(ctx, change)`.

---

## 2. Components

### 2.1 The map

```
┌──────────────────────────────────────────────────────────────────────────┐
│ CLIENTS                                                                  │
│ Claude Code · Claude · ChatGPT · Gemini · Cursor · Codex · web app · CLI  │
└────┬─────────────┬──────────────┬───────────────┬────────────────────────┘
     │ MCP         │ Quick Connect│ HTTPS/WS      │ git
     ▼             ▼              ▼               ▼
┌──────────────────────────────────────────────────────────────────────────┐
│ WORKER  (apps/worker)                                                    │
│   /health  /api/*  /mcp  /s/{token}/*  /ws  /docs  /hooks  /try  /       │
│   /oauth/*  /.well-known/oauth-*  (hosted OAuth, org hosts only)         │
│                                                                          │
│   src/context.ts        ProjectContext — the read-only handle            │
│   src/services/commit.ts  CommitService — THE WRITE PATH                 │
│   src/services/fts.ts     contentless FTS5 sync                          │
│   src/services/index-queue.ts  IndexJob contract + consumer              │
│   src/storage/*.ts        the only files that may hold a WriteToken      │
└──┬──────────┬──────────┬───────────┬─────────────────────────────────────┘
   ▼          ▼          ▼           ▼
┌──────┐  ┌──────┐  ┌──────────┐  ┌──────────────────────────────────────┐
│  D1  │  │  R2  │  │ Vectorize│  │ Durable Objects                      │
│      │  │      │  │          │  │ NoteDoc · IngestAgent · GitAgent     │
│notes │  │notes/│  │  chunks  │  │ RateLimiter                          │
│blocks│  │blobs/│  └──────────┘  └──────────────────────────────────────┘
│rel'ns│  │assets│
│chunks│  │export│                ┌──────────────────────────────────────┐
│_fts  │  └──────┘                │ QUEUES: signpost-index (+ dlq).      │
│prov. │                          │ git-sync and ingest are declared and │
│audit │                          │ nothing produces to them: GitAgent   │
└──────┘                          │ and IngestAgent serialise that work  │
                                  └──────────────────────────────────────┘
                                  ┌──────────────────────────────────────┐
┌──────────────────────────────┐  │ packages/core — ZERO Cloudflare      │
│ apps/web — React SPA (Vite)  │  │ imports. Parser, block identity,     │
│ docs · ask · review ·        │  │ chunker, retrieval fusion/packing    │
│ connect · account · settings │  └──────────────────────────────────────┘
└──────────────────────────────┘
```

### 2.2 The two halves, and the seam between them

`packages/core` is pure. No Cloudflare import, no `node:` import, no I/O, no clock it does not
receive as an argument. It holds the Markdown parser, block identity, the observation and relation
extractors, the chunker, and the fusion, decay, dedup and budget packer. It is where the product's
real value lives, and its purity is what makes a 95% coverage gate cheap instead of heroic.

`apps/worker` holds everything with a binding in it: the commit service, the storage repositories,
the FTS sync, the queue contract and the whole HTTP surface. Types that name a Cloudflare
interface live here and only here.

The seam is enforced, not aspirational:

- `packages/core/tsconfig.json` compiles with `types: ["node"]` for the ambient web globals only, and
  Biome forbids importing any `node:*` builtin from `src/`.
- `apps/worker/test/invariants/single-write-path.test.ts` scans `packages/core/src` for any write of
  any kind and fails if it finds one.
- `grep -rE "from ['\"](cloudflare:|@cloudflare/)" packages/core/src packages/core/test` returns
  nothing. The word "Cloudflare" appears in `packages/core` only inside comments explaining why it
  does not appear anywhere else.

### 2.3 `ProjectContext`

Every function that touches storage takes a `ProjectContext` as its first argument.
`resolveProjectContext(env, tenant)` builds one per request: the tenant middleware resolves the
request's hostname to an organization first, and the context carries that organization's project id.
A host that resolves to no organization gets the deployment's own default project. This is why a
per-request project costs no change in any query — see
[ADR 0004](./ADR/0004-project-context.md) and [`TENANCY.md`](./TENANCY.md).

What it carries is deliberately narrow:

```ts
interface ProjectContext {
  readonly projectId: string;          // the organization's project
  readonly projectName: string;
  readonly db: ReadableDatabase;       // Pick<…, "select" | "selectDistinct">
  readonly bucket: ReadableBucket;     // Pick<R2Bucket, "get" | "head" | "list">
  readonly vectors: VectorizeIndex;
  readonly ai: Ai;
  readonly queues: ProjectQueues;      // { index: Queue<IndexJob> }
  readonly writePolicy: WritePolicy;   // "open" | "review" | "locked"
}
```

`db` and `bucket` are read-only in the type **and** at runtime. Section 6 explains why that sentence
is the load-bearing one in this document.

---

## 3. The data model

```
Note        one Markdown file. Frontmatter + body. Stable `permalink`.
 └─ Block   an addressable unit: heading section, paragraph, list item, code block.
            Stable `block_id` that survives edits to other blocks.
     ├─ Observation   - [category] content #tag (context)
     └─ Relation      - rel_type [[Target]] (context)
Chunk       the retrieval unit, derived from blocks, embedded into Vectorize.
Proposal    a pending write that waits for a person's decision in the review queue.
```

**What reaches the review queue.** Each org has an **Agent writes** switch,
`org_routing.agent_writes`, and it starts `off` ([ADR 0022](./ADR/0022-agent-writes.md)). While it is
off, every agent change waits in Review for a person, a new doc included. While it is on, a connection
in write mode applies a new doc at a free address and an edit to the current version, and every
other shape — a delete, a stale or blind edit, a create at a taken address, a doc that replaces a live
one — waits. `changeShapeOf` in `services/commit.ts` is the one place that names the shape. A
connection that is not in write mode queues everything. The instance policy underneath is
`SIGNPOST_WRITE_POLICY: "conflicts"`, the shipped default;
[ADR 0017](./ADR/0017-conflict-only-review.md) says why its queue is for conflicts, and
[ADR 0016](./ADR/0016-direct-write-by-default.md) is the wave that made an assistant's write apply
at all. The switch is read on every request and never cached across requests. An upload that would
replace a doc also waits. A person's write is never queued by policy.

**Who may read a doc.** A doc is `open` or `private`
([ADR 0026](./ADR/0026-two-visibility-levels.md)). Every member reads an open doc. A private doc has a
list: `access:` in the doc or in a folder's `_folder.md` names the members and the `group:` addresses
that may read it, and `visibility: private` with no `access:` is the empty list
([ADR 0020](./ADR/0020-restricted-spaces.md) and [ADR 0023](./ADR/0023-sharing-people-and-groups.md)).
Owners pass every list, a nested folder takes the strictest one, and a person who fails a list gets
`404` and sees no count and no name. `CommitService` writes `notes.restricted` from the same parse as
the rest of the row, so the read scope finds private docs through a partial index. A connection reads
what the person who made it can read, with that person's denials, and it stops when that person
leaves the org. An anonymous reader of `/docs` reads `open` docs only.

**What a doc says about its author.** One rule, `displayProvenance` in
`packages/core/src/provenance/status.ts`, and every surface that shows a doc calls it through
`docProvenanceForRows` in `apps/worker/src/routes/api/waymark.ts`. The tree and the doc list call
that function directly. The doc reader, the `provenance` object on each `/api/recall` claim, Ask's
sources and the review queue call it through `docProvenanceByPermalink`, which looks the rows up by
address first. The claim's own `provenance.state` and
`trust` fields, and the MCP and Quick Connect `read` output, still carry the stored state and do not
pass through it. A human-authored doc never shows Unverified and never shows Accepted — both belong
to the assistant proposal path, and a person who writes a doc has proposed nothing. A person's own
write is stored `verified` by that person, so a doc only people wrote shows Verified.
`verified`, `stale` and `superseded` show on any doc, and the doc line names a verifier only
when the whole doc is verified, from its latest verification. In an org that does not use
verification ([ADR 0018](./ADR/0018-verification-is-an-org-choice.md)), no Verified, Accepted or
Unverified mark shows anywhere. The mark itself is `displayStatusFor` in the same core file:
`displayProvenance` calls it for a doc, Ask's source chip calls it for a claim, and `blockStatusFor`
in `apps/web/src/screens/kit/status.ts` calls it for one block.

**Addressing.** `kmem://note-permalink`, `kmem://note#blk_7f3a1c2d`, `kmem://note/depends_on/*`.
kmem writes only `kmem://`. Two aliases parse to the same refs: `sp://`, which every ref and audit
row used before 2026-09-17, and `memory://` for Basic Memory compatibility. No migration rewrote the
old rows or a person's Markdown, so every reader of a stored ref accepts both stored schemes
([ADR 0014](./ADR/0014-the-product-is-called-kmem.md), amendment). The grammar lives in
`packages/core/src/ref.ts` and is the authority — see §7.4 for why R2's idea of a legal key is not.

**R2 layout** (`apps/worker/src/storage/blobs.ts`):

```
notes/{folder}/{slug}.md      the canonical file. Mutable. This is the product.
blobs/{sha256}                content-addressed history: every version ever committed. Append-only.
assets/{id}/{filename}        images and attachments.
export/{timestamp}.tar.gz     full exports.
```

Every key an organization writes carries that organization's prefix. `tenantScopedKey` is the one
rule that applies it — [`TENANCY.md`](./TENANCY.md) §3.2.

**The D1 content tables** (`packages/db/migrations`): `notes`, `blocks`, `relations`, `chunks`,
`chunks_fts` (FTS5, contentless), `provenance`, `proposals`, `session_tokens`, `audit_log`. All of
them except `session_tokens` and `proposals` are derivable from R2 by replaying commits.

Beside them sit the tables that are not content and are not derivable from R2: `org_routing` (the
per-org switches, `agent_writes` and `verification` among them), `group_handle` (a group's fixed
address), the OAuth tables `oauth_clients`, `oauth_grants` and `oauth_codes` (`0020_oauth.sql` and
`0021_oauth_consent.sql`), and the cost tables `org_cost_daily`, `org_size_daily` and
`meter_rollup_runs` (`0026_cost_metering.sql`, [ADR 0025](./ADR/0025-cost-metering.md)). The cost
events themselves go to a Workers Analytics Engine dataset, `kmem_cost_events`, through metered
wrappers around each binding, and a nightly rollup reads them into D1.

---

## 4. The source-preserving Markdown model

This is the single most consequential design decision in the parser, and it is the reason invariant
I5 is true rather than aspired to. Full reasoning in [ADR 0003](./ADR/0003-source-preserving-markdown.md).

### 4.1 What it is

`parseNote(source)` **retains the original string verbatim** and returns blocks that carry
`range: {start, end}` offsets into it, measured in UTF-16 code units. `renderNote(note)` on an
unmodified note returns `note.source` — the original bytes, unchanged, because it still has them.

An edit is a **splice**: replace one block's range with new raw text and reparse. Nothing is ever
serialised back out of an abstract syntax tree.

Each block carries two representations, and both are needed:

- `raw` — the exact source slice. This is what a splice replaces.
- `text` — the normalised form. This is what gets hashed for identity, chunked, embedded and indexed.

### 4.2 Why, and what the alternative costs

The rejected alternative is what remark, unified and every Markdown editor toolchain do by default:
parse to an mdast tree, hold the tree as the working representation, serialise it back with a
stringifier on write.

A stringifier makes invariant I5 a permanent battle. Setext headings become ATX. `*` bullets become
`-`. Indentation normalises. Reference links inline themselves. Trailing whitespace disappears. Every
one of those is a diff in somebody's git history that they did not ask for, and you can chase them
with configuration and still lose on the hundredth fixture.

Keeping the source and splicing into it makes I5 true **by construction**: there is no round-trip to
get wrong, because there is no round-trip. The open-ended question "did the stringifier preserve this
edge case?" is replaced by two closed ones that are cheap to test exhaustively:

1. Do the block ranges **tile** the document exactly — sorted by `start`, contiguous, no gaps, no
   overlaps, covering `[0, source.length)`?
2. Does splice-then-reparse leave the ids of untouched blocks alone?

Both are asserted over every corpus fixture in `packages/core/test/fixtures/corpus/` and over 1,000
generated documents per run.

It also means kmem is safe to point at a repository it did not create. A file written by hand, by
Obsidian, or by another tool comes back exactly as it was found unless somebody deliberately edited a
block in it. A sync tool that silently reformats every file it touches is a sync tool nobody trusts
twice.

The cost is real and worth naming: the parser must be exactly as permissive as micromark about every
lexical boundary, because the two disagreeing by one byte is a data-loss bug rather than a formatting
bug. That is not hypothetical. A bare `\r` is a CommonMark line ending, the frontmatter fence scanner
once read only `\n`, and a splice therefore deleted a whole YAML header from a file whose fence ended
with a lone `\r`. Every line-ending primitive in `markdown/offsets.ts` is now CommonMark-correct, and
`packages/core/test/markdown/carriage-return.test.ts` holds it there.

### 4.3 Block identity

Ids are **derived, never written into the Markdown**. Storing them as HTML comments would make
identity trivial, and would also put machine noise into every file, breaking the first
non-negotiable: the Markdown has to read well in Obsidian, in a diff, and in a text editor.

```
blk_ + first 8 hex of sha256(headingPath.join("/") + "\0" + normalizedText + "\0" + occurrenceIndex)
```

`normalizeText` is the canonical form identity is computed over: strip U+200B and U+FEFF, apply NFC,
fold every whitespace run to one space, trim. It deliberately erases indentation, wrapping and line
ending style — rewrapping a paragraph at column 80 instead of 100 must not fork a block off its
history — and deliberately preserves U+200C and U+200D, which are grapheme-forming.

Deriving from content is not by itself stable, because the hash input includes `headingPath`: rename
a heading and every child block hashes differently. **`reconcileBlockIds` is what turns "derived from
content" into "stable across edits."** It carries ids across a reparse in two sweeps: an exact-match
sweep for blocks whose canonical text is unchanged, then a similarity sweep that recognises a lightly
edited block as an edit rather than as a delete plus a create. Where a key bucket holds duplicates,
the whole bucket is aligned at once — in order when the counts match, by a band-limited DP when they
do not — because deciding duplicates one at a time lets an earlier candidate at distance 1 beat a
later one at distance 0, and two identical blocks then exchange ids. That moves their provenance and
every inbound reference onto the wrong claim, so
`packages/core/test/blocks/identity.duplicates.test.ts` holds it shut.

The stored id must therefore always come from `reconcileBlockIds` on every write after the first,
never from `assignBlockIds`. `CommitService` does exactly that.

**Known rough edge, accepted:** moving a block under a different heading changes its id, because its
heading path is part of its identity. So does rewriting a block wholesale. Both are correct — an
edited claim is a different claim and its provenance should not silently carry over — but a block
that moves loses its inbound `kmem://note#blk_` references, and that is a real limitation, not a bug
to be surprised by.

---

## 5. The single write path

### 5.1 The shape

```
web edit ──────┐
MCP write ─────┤
WebMCP ────────┤
Quick Connect ─┼──► CommitService.commit(ctx, change) ──► R2: blobs/{sha}
git pull ──────┤              │                          D1 (one batch): notes, blocks,
folder ingest ─┘              │                            relations, chunks, chunks_fts,
                              │                            provenance, audit_log
                              │                          R2: notes/{folder}/{slug}.md
                              ▼
                        queue: IndexJob ──► embed → Vectorize
                                        ──► sweep forward refs       (idempotent backstop)
                                        ──► semantic dedup
                                        ──► hand off to GitAgent
```

Six doors, one path. `WRITE_SOURCES` is a closed union — `web`, `mcp`, `webmcp`, `quickconnect`,
`git`, `ingest` — and a seventh is refused rather than recorded.

### 5.2 The order of operations, and why it is that order

1. **Validate with zod.** Nothing has happened, so a malformed change costs one rejected promise
   and no side effect.
2. **Normalise the address, once.** `normalizeStoredPermalink` runs here and nowhere else in the
   write path, and it is idempotent (property-tested), so the note is stored at the address every
   read path resolves to.
3. **Bound the work before paying for it.** `MAX_MARKDOWN_LENGTH` (1,000,000 chars) then
   `MAX_INLINE_MARKERS` (20,000, checked by a linear scan *before* the parse), then
   `MAX_BLOCKS_PER_NOTE` (10,000), `MAX_RELATIONS_PER_NOTE` (1,000) and `MAX_CHUNKS_PER_NOTE` (2,000)
   immediately after it — before the first digest, the first query and the first byte written. A
   character count alone is not a bound on work: CommonMark's inline resolution is quadratic in the
   number of unresolved delimiters in a paragraph, so a legal 988,899-character note of 100,000
   wikilinks took 258 seconds, 105 of them inside the parse. The marker scan costs 25 ms over a
   megabyte and refuses the same document in 44 ms.
4. **Parse, extract, chunk, reconcile.** All pure, all in `packages/core`. Block ids reconcile
   against the note's *previous* blocks, which is what keeps editing block 3 from renaming 1, 2, 4
   and 5 and therefore what keeps their provenance rows and inbound refs pointing at the same claims.
5. **The blob first.** `blobs/{sha}` is immutable and addressed by its own content, so it is safe to
   write before anyone knows who wins: a blob no note references is inert, and writing it first is
   what makes the content recoverable from the bucket alone before anything derived from it exists.
6. **One D1 transaction.** `D1Database.batch()` is a single implicit transaction. The
   compare-and-swap, the note row, the blocks, the relations, the `chunks_fts` sync, the chunks, the
   provenance and the audit row all travel in it. That is what stops a failure halfway through
   leaving a note with three of its eight blocks — which would read as a successful, shorter note.
7. **The canonical file last**, and only by the writer whose swap took. See §7.
8. **Enqueue an `IndexJob`.** Embedding, forward-reference sweeping, dedup and git sync all happen
   after the commit returns.

### 5.3 Why the statements are assembled in `commit.ts` rather than called

`src/storage/*` exposes `replaceBlocksForNote`, `replaceChunksForNote` and friends, each running its
own `db.batch()`. Calling six of them in a row would be six transactions, and one commit must be one
transaction. D1 has no interactive transaction to hold open across round trips, so the only way to get
one is to build every statement first and hand them over together.

What `commit.ts` borrows from the repositories is therefore everything that is *not* the execution:
row projections (`toBlockRow`, `toChunkRow`, `toProvenanceRow`, `prepareAuditRow`), the
bound-parameter arithmetic, the id derivations, every read, and the deletion authoriser. No SQL is
spelled twice: the statements are compiled by Drizzle from the same schema, and only the two
`chunks_fts` statements are hand-written, because Drizzle cannot express a virtual table.

### 5.4 What the FTS5 sync has to be careful about

`chunks_fts` is contentless (`content=''`): SQLite keeps the inverted index and no copy of the text.
Three consequences drive the design.

- `DELETE FROM chunks_fts` is not allowed. The only way to unindex a row is the `'delete'` command,
  and it requires **the original column values** so FTS5 decrements the same postings it incremented.
  Hand it different values and it decrements postings it does not have: measured in workerd, the
  statement succeeds, the row stays findable by its old text, and the *next* `MATCH` against the
  table fails with `SQLITE_CORRUPT_VTAB` — for the whole table, not for that row.
- The rowid is the join key, so the FTS rowid must equal `chunks.rowid`.
- The sync is explicit inserts from `CommitService` in the same transaction as the chunk write,
  never triggers. A trigger would make the sync invisible; these statements make "the full-text index
  matches the chunks table" something a test can assert and a reviewer can read.

---

## 6. `WriteToken`: how invariant I1 is made structural

### 6.1 The problem

"Every mutation goes through `CommitService`" is an invariant enforced by convention, and an
invariant enforced by convention survives until the first hurried afternoon. The moment a route
handler needs to "just fix" a note, the shortest path is a direct write, review misses it, and the
audit log, the provenance rows, the FTS sync and the index queue all quietly stop being true for that
one write. Nothing fails. The knowledge base is simply wrong from then on.

So the rule was moved out of review and into the type checker.

### 6.2 The mechanism

Every mutating function in `apps/worker/src/storage/` takes a `WriteToken` as its second argument,
right after `ProjectContext`:

```ts
putBlob(ctx, token, content)
putCanonical(ctx, token, key, content)
deleteCanonical(ctx, token, key, tombstone)
```

You cannot call one without a token, and you cannot obtain a token unless you are the module granted
the mint. Four layers:

1. **Nominal, not structural.** `WriteToken` carries `#`-private fields, so no object literal is
   assignable to it and a direct cast fails too. The only escape left is `as unknown as WriteToken`,
   which needs a double cast — banned by repo rules and trivially greppable.
2. **Private constructor, escaped exactly once.** A `static {}` block inside the class body — the one
   place `new WriteToken(...)` is legal — hands the only reference to the constructor to a
   module-private `mint` that is never exported. The constructor additionally requires a
   module-private `unique symbol`, because `private` is a TypeScript-only construct that erases on
   the way to JavaScript and `Reflect.construct` would otherwise build a valid token at runtime.
3. **A closed allowlist for who may claim the mint.** `claimWriteAuthority` accepts the literal union
   `"commit-service" | "storage-conformance-test"`, checked at runtime as well as by `tsc` because
   the Worker ships as JavaScript. Adding a third member is a deliberate, visible diff, and it means
   adding a second write path.
4. **A brand check, not `instanceof`.** `Object.create(WriteToken.prototype)` satisfies `instanceof`
   and is typed `any` by `lib.es5.d.ts`, so `putBlob(ctx, Object.create(WriteToken.prototype), …)`
   would compile with no cast and no `@ts-expect-error`. `WriteToken.isMinted` tests for the private
   field instead, which nothing but the constructor can produce.

What it deliberately does *not* do: sniff `new Error().stack` to verify its caller (defeated by
bundling and inlining, and it converts a compile-time guarantee into a production-time one), and
throw on a repeat claim (module graphs legitimately evaluate twice).

### 6.3 How it was defeated by aliasing, and what now prevents that

**The token protocol guards the storage functions. It never guarded the binding they are built
over.** While `ProjectContext.db` was the full Drizzle handle, this was enough:

```ts
const database = ctx.db;                   // no error
await database.update(notes).set({ … });   // no error, no token, no audit row, no blob
```

One local variable defeated all four layers at once. It typechecked. It ran. It wrote no audit row.
And the static scan — anchored on the receiver names `db` and `bucket` — reported nothing, because
the receiver was now called `database`. `const store = ctx.bucket` did the same to R2.

Three things close it, and all three are necessary.

**(a) Narrow the type.** `ProjectContext.db` is `Pick<SignpostDatabase, "select" | "selectDistinct">`
and `ProjectContext.bucket` is `Pick<R2Bucket, "get" | "head" | "list">`. This closes aliasing *by
construction*, which is the property that matters: `const database = ctx.db` copies the **narrow**
type, so `database.update` does not exist however many times it is renamed. The raw handles live in a
module-private `WeakMap` and come back out only through `writableBindings(ctx, token)`, which demands
the same `WriteToken` every mutating function already demands.

**(b) Make the runtime half a frozen null-prototype object, not a `Proxy`.** It was a `Proxy` first,
trapping `get` and `has`, and a proxy that traps only those two hides nothing:

```ts
Object.getOwnPropertyDescriptor(ctx.db, "$client").value        // the raw D1Database
Object.getOwnPropertyDescriptor(ctx.db, "session").value.client // also the raw D1Database
```

Both reach the target directly, because `[[GetOwnProperty]]` and `[[OwnPropertyKeys]]` were
untrapped, and `PropertyDescriptor.value` is typed `any` by `lib.es5.d.ts`, so the escape typechecks
and lints clean. This was reproduced in workerd against real local D1: it wrote a row. Adding the two
missing traps would not have settled it either — a proxy must not lie about a non-configurable own
property, so the hole reopens on any target that freezes one.

What is on the context now is a **frozen, null-prototype object carrying nothing but the allowed
methods, each already bound to the real handle**. There is no target to reflect on, no prototype to
walk back up, and no standard way to recover a bound function's receiver. `Reflect.ownKeys(ctx.db)`
is exactly `["select", "selectDistinct"]`. Binding matters twice over: Drizzle's methods read
`this.session` and `this.dialect`, and R2's are host methods that throw `Illegal invocation` on a
foreign receiver, so an unbound copy would simply be broken — a loud failure rather than a hole — but
a bound copy is both correct and closed.

**(c) Allowlist reads by what they can express, not by what they are called.** `all` was on the
read allowlist because `services/fts.ts` needed a hand-written FTS5 `MATCH` that Drizzle cannot
express against a contentless virtual table, and it was described as "a read". But `db.all()` is
`D1Database.prepare().all()` and runs whatever statement it is handed:

```ts
await ctx.db.all(sql`delete from audit_log`);   // emptied the append-only log, no token
```

from a handle every read path holds. `all` is gone. `services/fts.ts` expresses its `MATCH` through
the query builder, which is *structurally* incapable of emitting anything but a `SELECT`.

**Defence in depth: the static scan.** `invariants/single-write-path.test.ts` greps the whole Worker
source for direct writes, for aliases of `ctx`/`context`/`env`, and for the write-token import. It
too has been defeated once and repaired: it blanks string contents before matching, so `env["BUCKET"]`
had already become `env["      "]` by the time the dotted patterns ran, and a real second write path
shipped green. Any *computed* access on `ctx`, `context` or `env` is now a finding, and every pattern
accepts `?.`. The scan is a backstop; (a), (b) and (c) are the guarantee.

**Six properties hold the facade shut, and one file asserts all six.**
`apps/worker/test/context.test.ts` runs inside workerd against real local D1 and R2: no write verb is
reachable on an aliased handle, no property-descriptor reflection yields the raw binding,
`Reflect.ownKeys` is exactly the two read verbs, the facade is frozen with a null prototype, a bound
read method exposes no receiver, and a spread clone of the context is refused by `writableBindings`.

---

## 7. Write serialisation

The full decision, the alternatives and the measurements are in
[ADR 0005](./ADR/0005-write-serialisation.md). The summary:

**Optimistic concurrency, with the note row as the serialisation point, and a strict ordering between
the two stores.** Two runtime facts constrain every possible answer: D1 has no multi-statement
transaction with rollback (`db.batch()` is atomic as a unit; two awaited statements are two
transactions), and R2 has no transaction at all and no conditional put on content. Two stores can be
*ordered*; they cannot be made atomic.

### 7.1 The compare-and-swap is real, and its guard asks the database

```sql
update notes set … where id = ? and content_sha = ? and deleted_at is …   -- the whole row state read
insert into notes (id, …) select null, … where changes() = 0              -- abort if the swap lost
```

A compare-and-swap that matches nothing does not abort a batch by itself — the remaining statements
would happily rebuild the index from the losing writer's content. The guard runs immediately after
the swap, inside the same batch: `changes()` reports the row count of the last completed statement,
so when the swap matched nothing it is zero, the select yields a row of nulls, `notes.id` is
`NOT NULL`, and D1 rolls the whole batch back.

The guard asks *how many rows changed*, not *does the row look like what I was about to write*. The
latter is false exactly when the winner wrote the same bytes in the same millisecond — which is what
a double-clicked save, a retried request, two ingest workers and an MCP retry all look like.

The swap restates the **whole** row state the commit read, `deleted_at` included, because a deletion
does not change `content_sha`: a swap that compares only content matches a row it has never seen, and
an update racing a delete silently revives the tombstone.

### 7.2 The blob is written first, the canonical file last, and only by the winner

`blobs/{sha}` is immutable, so writing it eagerly is safe. `notes/{folder}/{slug}.md` is mutable and
*is* the product, so whoever writes it last decides what the product says — and only the writer whose
swap took may write it. `publishCanonical` **converges** rather than assuming: each round re-reads
the note row, compares it against the sha R2 stored for the object, and writes the bytes **D1 names**,
fetched from `blobs/` when they are not the ones in hand. A writer whose put lands out of order
repairs it on its own way out; a writer that lost publishes nothing.

That ordering deletes a mechanism rather than adding one: nothing has to compensate the canonical
write when the transaction fails, because at that point nothing has written it. The only R2 residue
of a failed commit is an orphaned blob — history for a version that did not take, which the next
commit of those exact bytes reuses.

### 7.3 Everything two facts depend on is decided inside the transaction

Relation resolution is a statement in the batch, not a `select` before it:

```sql
update relations set to_note_id = (select id from notes where permalink = relations.to_permalink and deleted_at is null),
                     resolved   = case when exists (…) then 1 else 0 end
 where from_note_id = ?
```

Both "is the target live?" and "write resolved" happen in one serialised transaction, so a tombstone
can land before it or after it and never between. And when a commit *creates* a note, the same batch
carries the inbound sweep:

```sql
update relations set to_note_id = ?, resolved = 1 where to_permalink = ? and resolved = 0
```

which is exactly why invariant I3 holds with **no manual reindex and no queue involvement**:
committing the target resolves every waiting edge in the same transaction that creates it. The
`IndexJob` also carries `resolvePermalinks` so the consumer can sweep again, and that sweep is
idempotent — it only ever fills a column in — but it is a backstop, not the mechanism.

### 7.4 What cannot be raced is made structurally impossible instead

- **Provenance is keyed `(note_id, block_id)`**, and rows for blocks a commit removed are deleted in
  the same transaction. A content-derived block id can legitimately recur in another note; keyed on
  the block id alone, `ON CONFLICT DO NOTHING` handed the second note the first author's name, door,
  timestamp and verification state.
- **Permalink normalisation is idempotent**, shares its extension stripping with the R2 key builder
  so the address and the file it names cannot drift, and is applied at exactly one point.
- **A permalink must be an address, not merely a legal R2 key.** `validateObjectKey` deliberately
  permits `# ? & = % + : @`; `packages/core/src/ref.ts` forbids `/[\s#?*\\]/` in a segment. In that
  gap, `permalink: "backend/auth-decision#blk_deadbeef"` committed successfully and every audit row
  it wrote named an `sp://` ref that parses as *a block of somebody else's note*. `permalinkSchema`
  now requires the stored address to round-trip through `parseRef` back to a note ref naming itself.
- **Fan-out is bounded before it is paid for.** See §5.2, step 3.

### 7.5 What a caller sees

| Situation | Outcome |
| --- | --- |
| `expectedSha` supplied and stale | `CommitConflictError` naming both shas. Nothing written. This is the no-lost-update guarantee, and it is the mode the editor and MCP `edit` should use. |
| `expectedSha` omitted | Last writer wins. The swap still runs against the sha this commit read, and a lost race is retried from a fresh read up to `MAX_COMMIT_ATTEMPTS` (3). A write can be overwritten; D1, R2 and the index can never disagree. |
| `op: "splice"` | The read-modify-write happens inside the service, so a lost swap re-reads, re-applies the splice to the **new** content and retries. Two concurrent splices to different blocks of one note both survive. |
| Two concurrent creates of one permalink | One wins; the other violates `notes_permalink_idx`, rolls back, never reaches the canonical write, and gets a `permalink-taken` conflict. |
| A delete racing an update | One is refused, never both applied. The winner's transaction is the only one that happened, so the inbound edges agree with whether the note is still there. |

A canonical file that disagrees with `notes.content_sha` is a *classified* condition, not an alarm:
either a publish still in flight — the sha it holds is a version this service committed, and `blobs/`
proves it, so it is a race and is retried — or bytes nobody committed, which is the only case
reported as somebody writing to the bucket outside the write path.

---

## 8. The invariants, and the real failure each one prevents

Each invariant has a dedicated test. Never delete one. `CLAUDE.md` carries the same table in short
form, and `apps/worker/test/invariants/` holds the tests.

### I1 — No R2 or D1 write occurs outside `CommitService`

`apps/worker/test/invariants/single-write-path.test.ts`

**The failure it prevents.** Somebody adds a route handler that fixes a note directly. The note
changes in R2. D1 still holds the old blocks, the old chunks and the old FTS postings; no provenance
row records who did it; no audit row records that it happened; no `IndexJob` re-embeds it. Retrieval
then returns the old text of a note that no longer says that, attributed to somebody who did not
write it, and nothing anywhere reports an error. This is the failure mode that this entire product
category is known for, and it is why the rule is a type error rather than a convention. Mechanism in
§6.

### I2 — Deletion requires an explicit tombstone; a read or scan error never deletes

`apps/worker/test/invariants/no-delete-on-error.test.ts`

**The failure it prevents.** A sync loop cannot read the source directory. It treats "I could not
read it" as "it is empty". The reconciler concludes every note has been removed. The deletion path —
correct, tested, doing exactly what it was told — erases the knowledge base. Every step is
individually reasonable; the defect is that the deletion path accepted an *absence* as an
instruction. This has shipped in comparable products.

So a deletion cannot be expressed as an absence. It is a `Tombstone`, which cannot be constructed
(private constructor plus a module-private symbol, like `WriteToken`), is only ever returned by
`authorizeDeletion` — which requires the row to exist, to be live, and a caller-supplied
`expectedSha` to match — and which `deleteCanonical` refuses to act on unless the exact bytes it is
about to remove are already preserved under `blobs/{sha}`, verified against R2. A scan that returned
zero rows is refused outright, because an empty scan describes an unreadable source.

### I3 — Forward references enqueue resolution; they are never dropped

`apps/worker/test/invariants/forward-refs.test.ts`

**The failure it prevents.** Somebody writes "this depends on [[backend/redis]]" before that note
exists. A naive implementation drops the edge because it cannot resolve it. The note is written
later, and the link is gone — silently, with no error and nothing to notice. The graph then quietly
under-reports its own edges, which is worse than a broken link, because a broken link is visible.

Unresolved edges are stored with `resolved = 0` and `to_note_id = NULL`, keeping their context, one
row per distinct edge. They survive updates to the source note, chunk regeneration, embedding, a
refused commit, and the source note being deleted and revived. Deleting a target *unresolves* inbound
edges rather than deleting them. The only thing that removes an edge is the line that stated it going
away.

### I4 — Every committed chunk is embedded or appears in the index-lag metric

`apps/worker/test/invariants/index-coverage.test.ts`

**The failure it prevents.** A chunk is written but never embedded — the queue send failed, the
consumer crashed, the embedding model changed. Retrieval never returns it. `/health` says the index
is complete. The knowledge base has a hole in it that nobody can see, and the only symptom is an
assistant that "does not seem to know about" something that is definitely written down.

So a chunk is written unembedded and counted in the lag metric until something records the embedding.
The metric never rounds a nearly-complete index up to complete, measures lag from the oldest waiting
note rather than the newest, reports full coverage for an empty corpus, and never reports a negative
lag when the clock disagrees with the row. An enqueue that fails still commits and still counts the
chunks it could not hand off. A re-commit returns a rewritten chunk to the backlog and leaves an
unchanged chunk embedded, so a one-word edit is not a full reindex.

### I5 — Markdown → parse → render → Markdown is byte-identical

`apps/worker/test/invariants/roundtrip.test.ts`, plus `packages/core/test/roundtrip/*`

**The failure it prevents.** A user's file comes back with `*` bullets turned into `-`, setext
headings turned into ATX, and trailing whitespace stripped. Their next `git pull` is a diff touching
every file they did not edit. They stop trusting the tool with their repository, which is the only
thing it was for. Worse, if the mangling is lossy rather than cosmetic — a fence boundary read
differently from the way micromark reads it — content disappears. §4.2 names the case that did it.

Held to two halves: `renderNote(parseNote(s)) === s`, which is true by construction and therefore
proves only that the parser did not mangle the string it was handed; and
`renderBlocks(parseNote(s)) === s`, which reassembles the document from the block ranges alone and
*can* fail. When the second fails, the tiling is broken and a splice would drop or duplicate bytes.

### I6 — A revoked or expired token is rejected on every endpoint

`apps/worker/test/invariants/token-lifecycle.test.ts`

**The failure it prevents.** Somebody leaves the team, or a Quick Connect URL turns up in a
transcript. The link is revoked. It keeps working for another five minutes because a lookup is
cached, or on one endpoint because that route checks the token differently. A credential that is
revoked-but-working is worse than one that was never revoked, because everybody believes the problem
is handled.

Two halves hold it. The storage half: a token is stored as a SHA-256 only, the lookup returns nothing
for a revoked or expired row, and a dead token cannot widen its own scope. Since 2026-09-16 the lookup
also reads the membership of the person who made the connection, on every request: no `member` row
means `revoked`, whatever the expiry says ([ADR 0020](./ADR/0020-restricted-spaces.md)). The same test
proves that turning an org's Agent writes switch off takes effect on the next request. The endpoint half: the test
drives the real authenticator over every endpoint, so a route cannot check a credential its own way.
One implementation of the lookup is the point of both halves.

### I7 — A note a session may not read appears in no response at all

`apps/worker/test/invariants/read-scope.test.ts` (allowlist in `read-scope-allowlist.ts`)

**The failure it prevents.** A private note leaks its own existence through a search
result, a gap row, a briefing table or an error body. The reader cannot open it and now knows it is
there, which is the half of the secret that matters. This is why a note outside a credential's scope
answers `404`, never `403`. The same rule covers a doc or a folder that an `access:` list closes to a
member: the read scope carries the denied folders and docs, and every surface that honours one honours
the other.

### Three more scans, each with its own test

- **`invariants/permalink-address.test.ts`** — a stored permalink must round-trip through `parseRef`
  back to a note ref naming itself. Prevents an audit row that names somebody else's block (§7.4).
- **`invariants/provenance-ownership.test.ts`** — a provenance row belongs to one `(note_id,
  block_id)` pair. Prevents an agent's write being reported as a named human's (§7.4).
- **`invariants/one-slug-rule.test.ts`** — exactly one rule turns a title into an address, and it
  lives in `@signpost/core`; the test scans both apps for a second implementation. Prevents the
  drift where two slug rules mint two addresses for one note and a stored permalink stops
  resolving.

The same directory holds the scans that enforce the same style of rule elsewhere: one hostname rule,
one id rule, one acceptance rule, tenant scope, public surfaces and the clamp guards. Read the
directory before you add a rule that already has an owner.