---
title: "ADR 0003 — The parser preserves the source; it never reconstructs it"
type: decision
tags: [signpost, adr, decision]
proposed_by: tok_bc09b340db294c709763789154d8113d
model: kmem-docs-publisher
extracted_at: "2026-10-08T22:37:29.125Z"
---
# ADR 0003 — The parser preserves the source; it never reconstructs it

**Status:** accepted
**Date:** 2026-08-08

## Decision

`parseNote(source)` retains the original string verbatim and returns blocks that carry
`range: {start, end}` offsets into it. `renderNote(note)` on an unmodified note returns
`note.source` — the original bytes, unchanged, because it still has them.

An edit is a **splice**: replace one block's range with new raw text, then reparse. Nothing is ever
serialised back out of an abstract syntax tree.

Block ids are derived rather than stored: `blk_` plus the first eight hex characters of
`sha256(headingPath + "\0" + normalizedText + "\0" + occurrenceIndex)`, reconciled across reparses
so that editing block 3 does not change the ids of blocks 1, 2, 4 and 5.

## Alternative rejected

Parse Markdown to an mdast tree, hold the tree as the working representation, and serialise it back
with a stringifier when writing. This is what remark, unified and every Markdown-based editor
toolchain do by default.

A second alternative, also rejected: write block ids into the Markdown as HTML comments
(`<!-- blk_7f3a1c2d -->`) so that identity is stored rather than derived.

## Why

**On round-tripping.** Invariant I5 requires that Markdown parsed and rendered comes back
byte-identical, over at least 200 fixtures including CommonMark edge cases, nested lists, tables,
code fences containing Markdown, HTML blocks, footnotes, mixed line endings, Unicode and emoji, plus
a thousand fuzzed documents. A stringifier makes that a permanent battle: setext headings become
ATX, `*` bullets become `-`, indentation normalises, reference links inline themselves, trailing
whitespace disappears, and every one of those is a diff in someone's git history that they did not
ask for. You can chase it with configuration and you will still lose on the hundredth fixture.

Keeping the source and splicing into it makes I5 true **by construction**. There is no round-trip to
get wrong, because there is no round-trip. The tests then stop being "did the stringifier preserve
this edge case" — an open-ended question — and become two closed ones:

1. Do the block ranges tile the document exactly: sorted by `start`, contiguous, no gaps, no
   overlaps, covering `[0, source.length)`?
2. Does splice-then-reparse leave the ids of untouched blocks alone?

Both are cheap to test exhaustively and both fail loudly.

It also means Signpost is safe to point at a repository it did not create. A file written by hand,
by Obsidian, or by another tool is returned exactly as it was found unless someone deliberately
edited a block. A sync tool that silently reformats every file it touches is a sync tool nobody
trusts twice.

**On derived ids.** Storing ids in HTML comments would make identity trivial and stable. It would
also put machine noise in every file, which breaks the first non-negotiable: the Markdown is the
ground truth, and it has to read well in Obsidian, in a diff, and in a text editor. Deriving the id
from heading path plus normalised text plus occurrence index costs a reconciliation pass and buys a
file that looks like something a person wrote.

## Consequences

- `Block` carries both `raw` (the exact source slice, what a splice replaces) and `text` (normalised,
  what gets hashed, chunked, embedded and BM25-indexed). Both are needed; neither is redundant.
- Frontmatter is a block with a range like any other, so it tiles with the rest and can be spliced.
  A note with no frontmatter has an empty range at offset 0.
- Ranges are measured in UTF-16 code units — JavaScript string indices — so `slice` is exact.
- Block ids can collide when two blocks have identical text under the same heading. The occurrence
  index in the hash input is what disambiguates them, and it is why the reconciliation pass has to
  be order-aware.
- Ids change when a block's text or its heading path changes. That is correct: an edited block is a
  different claim, and its provenance should not silently carry over. Moving a block between
  headings also changes its id, which is a known and accepted rough edge.
- The editor round-trip gate (phase 4) is narrower than the parser gate (phase 1) and deliberately
  so: BlockNote is a different serialiser with its own vocabulary, and holding it to 200 arbitrary
  CommonMark fixtures would be holding it to a promise Signpost's own parser keeps by a mechanism
  BlockNote does not have.