The desktop app catches up with the corpus

Why Care?

Earlier today the corpus learned two new tricks: every PDF moved into a content-addressed store — one copy, compressed, fetched on demand — and a change feed started rendering what we did and why in language a client could read.

Both were true only in the terminal. The desktop app knew nothing about either. A source whose PDF now lived in bin/ looked, on screen, exactly like a source with no PDF at all.

This is the entry where the screen catches up. It is also the entry where opening the app turned out to be reading the entire corpus to draw a header, and we can say exactly how much that cost.

What's New?

• Sources carry their binary. Each row says whether its PDF is on this machine, how big the working copy is, and whether it was optimized — with an Open or Download link accordingly.

• A "What changed" tab. The change feed, rendered from the same records corpora changes prints, so the screen and the CLI can't drift into two notions of what happened.

• corpora fetch — get any binary back by key, or by handing it the wrapper that names one.

• Opening the app got ~25× faster. /api/meta went from 20.6 seconds to 0.8; a fifty-row page from about 20 seconds to 2.8.

The blank window

The wiring went in, the app launched, and this happened:

Starting the backend…

Nothing else. Forever, as far as anyone watching was concerned.

The sidecar was fine — /healthz answered instantly. The Rust shell had spawned Python exactly as designed. What we had was a healthy backend and a window that would not paint.

/api/meta was taking 20.6 seconds.

It returns four things: a label, a count, a domain list, and a writable flag. To get the count and the domains it called list_sources, which read all 845 markdown files out of Cloudflare R2, parsed each one, and threw away everything except two numbers.

Both facts live in the key.

def list_domains(store, prefix=""):
    keys = [k for k in store.list(prefix) if k.endswith(".md")]
    return len(keys), sorted({_domain_of(k) for k in keys})

One list() call. No file body opened. 20.6s → 0.8s.

Which only moved the problem

With meta fast, the first /api/sources call inherited the cost — because it too read every file, then sliced the result down to a page.

So an unsearched listing now pages over keys before reading anything. Fifty rows costs fifty reads instead of 845.

That trade has a real edge worth naming: ordering switches from fetched_at (inside the file) to the filename's date prefix. The naming convention writes that prefix from fetched_at, so the two agree — and a file predating the convention sorts by name, which is an honest fallback rather than a confidently wrong date. A search still reads everything, because it has to, and a search is a deliberate act rather than a page load.

And that left the latency

Fifty sequential GETs against R2 at roughly 140ms each is seven seconds of waiting rather than working. The page now reads through a thread pool, with order restored afterward.

Under the hood

The tests assert reads, not seconds

listing = list_sources(store, limit=10)
assert listing.total == 40   # the count is still honest
assert store.reads == 10     # ...but only ten files were opened

A performance promise measured in wall clock is a promise that passes on a fast laptop and rots quietly. Counting reads is the thing we actually mean.

Three schemas, met where they are

Two rows rendered 0 B. It turns out reach-edu carries three shapes of binary_asset: fetched PDFs use size_bytes and content_type; uploaded ones were recorded with bytes / original_bytes / compressed by an earlier pipeline that did its own compression; and today's migration added binary_key / optimized.

We modelled all three rather than migrating them. Rewriting a client's frontmatter to tidy a synonym is exactly what the pointer module was built to refuse.

One read that writes

/api/binary is allowed on a read-only server. Populating a local cache from an immutable, content-addressed object cannot alter the corpus — so this is stated in the spec rather than left to be rediscovered and "fixed" by gating it behind --writable later.

The bug that was almost invisible

create_app hardcoded the machine-level cache. A test evicted a binary, asked the app about it, and was told present — because the app was reading whatever this laptop happened to have downloaded. The cache is now injectable.

That is the third false-green of the same shape in one day, and the pattern is worth stating plainly: a test that does not control its environment is testing the environment.

What's Next

• Wire capture's binary path into the screen — new captures already file into bin/ correctly, but the capture form doesn't say so.

• Kill the last cold cost. 2.8s to first paint is livable, not good.

• The not_downloaded affordance is untested by a human. Every binary in reach-edu is currently cached locally, so nobody has yet clicked Download PDF on a cold machine.

Related

• changelog/2026-08-22_02.md — the change feed this surfaces

• changelog/2026-08-22_03.md — the bin/ store this makes visible

• context-v/specs/Browse-Corpus.md — behaviours 10–13, and the read-count promises

• context-v/specs/Binary-Ingest-And-Bin-Store.md — where not_downloaded is defined as a state rather than an error