--- title: "Ask" type: reference tags: [signpost, docs, reference] proposed_by: tok_bc09b340db294c709763789154d8113d model: kmem-docs-publisher extracted_at: "2026-10-08T22:27:26.736Z" --- # Ask Ask is the second of the four destinations in the navigation: Docs, Ask, Review and Connect. A person types a question in plain words. Ask answers in two to six sentences of prose. Under the answer it shows the docs the answer rests on, as chips, and up to three follow-up questions. When the docs do not cover the question, Ask says so. It is not a search screen. It shows no score, no rank, no chunk id and no `kmem://` ref inside a sentence. That rule is the whole point of the feature, and three things hold it: - The system prompt bans the register and names the phrases it bans. - The stream splitter holds back the machine-readable tail. - The service checks the answer's own `cited` list against the docs retrieval returned. ## 1. Ask and the in-app assistant are two features `POST /api/assistant/ask` answers as an **agent principal**. It builds a synthetic session with `author_kind: agent`. It runs the MCP verbs through the same dispatcher `/mcp` uses, and it can propose a write. It hands back a tool trace, a token budget and a claim list. Its reader is a person who audits an assistant. `POST /api/ask` answers as **the person themselves**. It reads with their own `ReadScope` (invariant I7), runs retrieval (and a bounded research loop when the first pack is weak), writes nothing, and hands back prose plus the docs it rests on. Its reader is a person who wants to know a thing. Do not fold them. The first casualty would be Ask's own rule. ## 2. The wire ### `POST /api/ask` Session cookie, on an org host. A request with no session, or one on the apex, gets the usual refusal. The per-person rate limit applies: 600 requests a minute, and one of the ten concurrency slots for as long as the answer runs. No plan and no monthly budget caps Ask (owner, 2026-10-08). Every answer is recorded in the spend ledger. ``` body { "question": string (1..2000), "history": [{ "role": "user"|"assistant", "text": string }] (0..8, optional), "conversation_id": string (optional, client-minted) } ``` With `Accept: text/event-stream` the answer is server-sent events, in this order: ``` event: sources data: { "sources": [ { "ref": "kmem://...", "permalink": "...", "title": "...", "folder": "...", "heading": string|null, "status": "verified"|"accepted"|null, "written_by": string|null, "uploaded_by": string|null, "snippet": "..." } ] } event: delta data: { "text": "..." } event: done data: { "answer": string, "grounded": boolean, "cited": ["kmem://..."], "followups": [string], "gap": null | { "reason": "not-covered" | "search-limited", "suggested_title": string }, "model": string, "truncated": boolean } ``` `sources` always arrives first, and it arrives even when nothing matched — an empty list is a fact the screen draws. Zero or more `delta` events follow. Exactly one `done` closes the stream. `gap.reason` says why the docs did not answer. `not-covered`: the search ran in full and nothing answers it, so the screen offers "Write it down". `search-limited`: part of the search did not run — nothing is embedded yet, or vector or keyword search was unavailable — so a doc that answers it may have been missed, so the screen offers no door to write a doc. There is no `withheld` reason since [ADR 0027](./ADR/0027-verification-is-tracking.md): no doc is held back from Ask. A screen that still reads `withheld` from an older Worker draws it as `not-covered`. `truncated` is `true` when the model stopped at its output cap; the answer then ends with an ellipsis, so the cut shows in the words. An `error` event replaces the rest when something fails, and the stream then closes: ``` event: error data: { "cause": "...", "fix": "..." } ``` Retrieval failing produces `error` alone. The model failing produces `sources` then `error`, because by then the chips are already correct. So does a model that wrote no answer at all: that is a failure the person can retry, never a gap in their docs. Stop cancels the stream. The Worker stops reading the model at the next event, logs `ask_stream_cancelled` at `warn`, and meters the answer call as one call whose cost was not reported. With any other `Accept` the same turn comes back as one JSON object: every field of `done`, plus `sources`. A turn that produced no answer at all is `503 { cause, fix }` instead. A refused rate limit is `429` with the usual body. ### `GET /api/ask/starters` ``` 200 { "starters": [ { "question": string, "ref": "kmem://..."|null } ] } ``` Three to five questions. An organization with docs gets questions drawn from the titles of the twelve docs it updated most recently. Each question links to the doc it came from. When the model call fails, or answers with fewer than three usable questions, the set is topped up with one plain question per recent doc — `What does "Session expiry" say?` — linked to that doc. An organization with no docs gets none: every question it could be offered would end in "Not in your docs yet". This is the one `GET` under `/api` that is rate limited, because it is the one `GET` that spends a model call. ### An example transcript ``` POST /api/ask Accept: text/event-stream {"question":"How long do sessions last?","conversation_id":"conv_7"} event: sources data: {"sources":[{"ref":"kmem://backend/session-expiry#blk_1a2b3c4d","permalink":"backend/session-expiry","title":"Session expiry","folder":"backend","heading":"Timeouts","status":"verified","written_by":"Ankur","uploaded_by":null,"snippet":"A session expires 24 hours after it is issued."}]} event: delta data: {"text":"Sessions expire "} event: delta data: {"text":"24 hours after they are issued, and a refresh extends them by another 24."} event: done data: {"answer":"Sessions expire 24 hours after they are issued, and a refresh extends them by another 24.","grounded":true,"cited":["kmem://backend/session-expiry#blk_1a2b3c4d"],"followups":["What happens to an agent's session link?"],"gap":null,"model":"@cf/zai-org/glm-5.3-flash"} ``` ## 3. How a turn runs 1. **Retrieve.** The same pipeline `POST /api/recall` runs, with the same config and the caller's own `ReadScope`. Two doors, one implementation: a screen that ranked differently from the assistant connected to the same instance would be two products. Every doc text the person may open is a source, an agent's unverified text included. In an org that uses verification, verified text ranks above accepted text, which ranks above unverified text; in an org that does not, every claim ranks the same ([ADR 0027](./ADR/0027-verification-is-tracking.md), [ADR 0018](./ADR/0018-verification-is-an-org-choice.md)). A change that waits in Review is not doc text, so it is never a source. A short question is searched together with the one before it only when it leans on it: it points back with a pronoun ("Who decides it?") or names nothing a doc could be about ("Why?"). "What is the PTO policy?" stands alone, whatever came before it. 2. **Trim.** At most 12 claims across at most 6 docs, in rank order. The token budget bounds how much text a pack carries. This bound is on how many chips the screen draws. Fourteen chips is a search result. 3. **Research, when the first pack is not enough.** `askAgent` may call the model to choose a read-only next step: search again, open a doc, list a folder, or answer. Caps are in `ASK_LIMITS` (`services/ask/agent.ts`): six steps, eight docs, and separate times for research and for the answer. When the first retrieval already has sources, reports no gaps, and ran its search in full, the turn skips that action call and writes the answer at once — the call would only have said "answer". A weak first pack still researches, and a search or open the model asks for still runs. `ask_agent_answer` logs `research: "skipped" | "ran" | "none"` so a turn with `tool_steps: 0` is not ambiguous. 4. **Ask.** One streamed completion. The prompt lists the docs as numbered sources, each with a title, a folder and the passages retrieval kept from it, whole. The chip's short snippet is for the chip: while the model read it instead, it answered as if a doc stopped where the snippet did. Numbers rather than refs: a model cannot smuggle a number into a sentence. When part of the search did not run, the prompt carries a search note, so the model does not tell the person their docs lack something they may hold. 5. **Split.** The prose streams to the person. After a line of three hyphens the model may write one JSON object, `{"cited": [n...], "followups": [...], "gap": ...}`, and nothing after the sentinel ever reaches the screen. 6. **Finish.** A model that wrote no tail costs one short second call through the `askFollowups` row, which returns the citations as well as the follow-ups and the gap. A small model skips the tail often. `grounded` is `true` only when a citation judgement — the tail's or the second call's — named a source that was in `sources`. A number the model invented resolves to nothing, and the parser drops it. When neither call judged the citations, the turn shows neither chips nor the not-covered card. A model that wrote no prose at all ends the turn with `error`. 7. **Log.** One `warn` line, `ask_answer`, with the model, the source count, the cited count, `grounded`, and the call's tokens and neurons. `warn`, because `requestLogging` drops `info` on every deployment that does not configure one. The agent path also logs `ask_agent_answer`. ## 4. The models config `apps/worker/src/ai/models.ts` is the one table of every place this product calls a model. Each row names its provider, its model, its output cap, its temperature, whether thinking is on, and what it is for. | Place | Purpose | Output cap | Temperature | Thinking | | --- | --- | --- | --- | --- | | `ask` | Answers a person's question in prose from the docs retrieval found | 4,000 | 0.2 | on | | `askFollowups` | Writes the follow-up questions and the gap when the answer carried no tail | 2,000 | 0.4 | on | | `askStarters` | Turns the most recently updated doc titles into starter questions | 2,000 | 0.4 | on | | `assistantGrounded` | Composes the in-app assistant's cited answer from a context pack | 4,000 | 0.2 | on | | `assistantGeneral` | Writes the in-app assistant's uncited "outside the knowledge base" aside | 2,000 | 0.2 | on | | `importMemory` | Groups the entries of another AI's memory export into proposed folders and docs | 16,000 | 0.2 | off | Every row defaults to `@cf/zai-org/glm-5.3-flash` on Workers AI, through the `AI` binding this Worker already has. A deployment needs no third-party credential to answer a question. Most rows turn thinking **on** (`thinking: true`, sent to Workers AI as `chat_template_kwargs: { enable_thinking: true }`). The plan then arrives in `reasoning_content`, which the adapters never read, so `content` opens with the answer. The reasoning filter (`apps/worker/src/ai/reasoning.ts`) still drops any tagged reasoning section that lands in `content`, and it does **not** hold the start of the reply: the answer streams from its first delta. `importMemory` is the exception: thinking off, because with it on the reasoning spent a whole cap and left the JSON empty. With thinking off the plan can sit untagged in `content`, so that row's filter still holds the start until a closing tag, the hold fills, or the reply ends. On 2026-09-12, Ask answers with thinking left off and a 700-token cap used the whole cap on a plan and wrote no prose. The starters call did the same at 300. The caps and the thinking switch above are what replaced that. ### Overriding one row One variable per row, holding `provider:model`: ```sh wrangler secret put SIGNPOST_MODEL_ASK # "openai:gpt-5.6-luna" wrangler secret put SIGNPOST_MODEL_ASK_FOLLOWUPS # "workers-ai:@cf/zai-org/glm-4.7-flash" wrangler secret put SIGNPOST_MODEL_ASK_STARTERS wrangler secret put SIGNPOST_MODEL_ASSISTANT_GROUNDED wrangler secret put SIGNPOST_MODEL_ASSISTANT_GENERAL wrangler secret put SIGNPOST_MODEL_IMPORT_MEMORY ``` The four providers are `workers-ai`, `openai`, `anthropic` and `openrouter`. The last three each need their own key: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `OPENROUTER_API_KEY`. A row that names a provider with no key refuses, and the message names the variable to set. An override that names a provider this build has no adapter for refuses the same way. Ask ignores neither one quietly. The model half may itself contain a colon, so the value splits on the **first** colon only. The row's own output cap and temperature survive an override: the variable chooses a model, not a budget. This build retires `SIGNPOST_ASSISTANT_PROVIDER` and `SIGNPOST_ASSISTANT_MODEL`. The code still reads them for one release. They map onto the `assistantGrounded` row, with the exact model each one used to imply. Delete that branch in `services/assistant/env.ts` one release after this one. ## 5. What an answer costs Cloudflare bills Workers AI in neurons, at $0.011 per 1,000 (). For `@cf/zai-org/glm-5.3-flash`, read from the same page on 2026-09-11: | | Per million tokens | In neurons | | --- | --- | --- | | Input | $0.150 | 13,636 | | Output | $0.500 | 45,455 | `apps/worker/src/inference-cost.ts` holds that table with its source and its date, and `services/spend/ai-spend.ts` turns a reported token count into a ledger row with `usd_source: "table"`. A response that reports its own neuron figure keeps it. The table prices only the calls that reported no figure. For a model the table has no row for, the row carries the real token counts, zero dollars and `usd_source: "none"`. A `warn` line then names the model, so an operator can add a row. Nothing guesses a rate. One answer costs a `recall`, about 1.759 neurons, plus one completion. The model reads the passages retrieval kept, and the pack that holds them is capped at 4,000 tokens, so a grounded answer over six docs sends at most about 4,500 input tokens with the prompt and writes roughly 120 output tokens. That is at most about **70 neurons**, or **$0.0008**, so one answer costs up to about forty `recall` calls. Most packs are smaller than the cap. A follow-up question also sends the conversation so far: at most the last 8 turns, and at most 4,000 tokens of them, keeping the newest turns that fit (`MAX_ASK_HISTORY_TURNS` and `MAX_ASK_HISTORY_TOKENS` in `services/ask/types.ts`). A long conversation can therefore add up to about 55 more neurons of input to one answer. Before that cap, eight long turns made one answer cost about four times a typical one; [`PRICING_ANALYSIS.md`](./PRICING_ANALYSIS.md) has the measurement. **That figure is arithmetic from the table above. Nobody has measured it.** To replace it, ask one question against a deployment. Then read `ask_answer`'s `input_tokens` and `output_tokens` off `wrangler tail --format json`. ## 6. Where the code is | Path | What is there | | --- | --- | | `apps/worker/src/ai/models.ts` | The table, `modelFor`, `providerFor`, and the test seam | | `apps/worker/src/ai/workers-ai.ts` | The `AI` binding as an `AiProvider`, with `stream()` | | `apps/worker/src/ai/sse.ts` | Reading a streamed Workers AI answer. Pure, over a `ReadableStream` | | `apps/worker/src/ai/reasoning.ts` | Drops reasoning from every model reply; holds the start only when thinking is off | | `apps/worker/src/services/ask/retrieve.ts` | The pack, trimmed, and the doc chips | | `apps/worker/src/services/ask/prompt.ts` | What the model is told, for all three Ask calls | | `apps/worker/src/services/ask/parse.ts` | The stream splitter and the tail reader | | `apps/worker/src/services/ask/agent.ts` | The research loop, the skip rule, and the caps | | `apps/worker/src/services/ask/starters.ts` | The starter questions | | `apps/worker/src/services/ask/ask.ts` | One turn, as events | | `apps/worker/src/routes/api/ask.ts` | The two endpoints | | `packages/core/src/ai/ports.ts` | `AiProvider`, `AiStreamingProvider`, `supportsStreaming` | ## 7. Three things a later change must not break **A byte of the JSON tail must never reach the screen.** The model writes prose, then a line of three hyphens, then JSON. That sentinel arrives in one of three ordinary shapes, and a naive splitter leaks on all three: - split across two chunks; - with its newline in a chunk of its own; - with the whole tail in one delta. `parse.ts` holds back the shortest suffix that could still grow into a sentinel. `test/services/ask/parse.test.ts` feeds it an answer one character at a time, and asserts the prose contains no brace. The mirror of that failure is an answer cut in half. A model asked about a delimiter writes three hyphens on a line in the prose itself. A splitter that trusts the first such line then shows the reader a sentence that stops. The prompt asks the model not to write that line. The splitter does not depend on the model obeying. A tail with no `{` in it was never a tail. The splitter then puts the sentinel, and everything after it, back into the prose. A third case needs the whole answer. A model that writes a lone `---` in the prose **and** a real JSON tail after it — a rule above a list, say — used to lose the sentences between the two, and all of the answer when the rule opened it. Once the stream ends, the last three-hyphen line with JSON after it is the real sentinel, and what came before it goes back into the prose. That text arrives at the end in one piece rather than streamed, which is the price of not buffering every answer in case this happens. **The route must not load the service at module scope.** `routes/api/index.ts` imports `routes/api/ask.ts` to mount it, so a static import there joins every `/api` request's cold start. `services/ask/retrieve.ts` reaches the retrieval pipeline, the trust service and the auth store. Loading that graph eagerly added about ten seconds to `test/routes/api/spend.test.ts`, which was enough for its seventy sequential writes to cross the rate limiter's fixed sixty-second window. A test about the rate limiter then went red on a change that never touched it. The handlers use `await import(...)`, the shape `routes/api/auth.ts` already uses, and a source scan in `test/routes/api/ask.test.ts` refuses a static one. **A doc the caller may not read must contribute no chip.** The retrieval ports filter by `ReadScope` before the pipeline ranks anything. The title lookup in `retrieve.ts` passes the same scope, because a title is content. A later edit is most likely to drop that second gate, so `test/services/ask/retrieve.test.ts` plants a claim for a private doc and asserts no chip comes back.