Lossless Changelog
product

The context-v tier sweep, and the corpus goes to Chroma Cloud

Every context-v document across ~40 repos swept for frontmatter conformity, then re-indexed into Chroma and pushed to the cloud.

Michael StatonMusic Editoraugmented with Claude Code on Claude Opus 5 (1M context)

Why Care?

A retrieval corpus is only as good as the metadata it is indexed on. The changelog tier was swept in a prior pass; this one finishes the job across context-v/, the larger and messier half — full of stubs, drifted dates, and documents that had never had frontmatter at all. The corpus that feeds Chroma is deliberately wider than the swept tier and now stands at 1,166 files. With that settled, the corpus was re-indexed and, for the first time, pushed to Chroma Cloud so retrieval is not bound to one laptop.

What's New?

The sweep. Every context-v/ document across the tree was brought to frontmatter conformity: identity (site_uuid, hex_code) minted where missing and never overwritten where present, editorial dates reconciled against filesystem and git evidence, ledes and summaries authored, and a publish decision recorded per document. Thirty-eight files that had no frontmatter block at all got one. Eighteen unparseable YAML blocks were repaired. A confidentiality screen ran over the public repos, genericizing client and portfolio names where naming them added nothing.

Accuracy, not just conformity. The date work was the substantive part. Thirteen date-resolution chains predated the editorial-date convention and silently resolved to nothing, leaving sixteen documents rendering blank dates on the sites. Client-engagement documents kept their original dates rather than being re-stamped, because the dates are evidence of when the work happened.

Robustness. Nine hand-rolled frontmatter parsers across the splash sites mishandled YAML block scalars (>-, |-), turning a folded lede into the literal string >- — one site shipped that to production in a meta tag. All nine were patched and proven functionally identical, which makes the consolidation onto LFM provably safe. Logged as an issue in astro-knots.

Chroma. The corpus was re-collated (it had drifted a month) and re-ingested: 10,179 chunks in context-vigilance-corpus, 522 in lossless-changelog.

Chroma Cloud. All 10,179 chunks now live in the lossless-corpus cloud database. Getting there required two fixes to ingest-to-chroma.py, both of which had been latent:

  • chunk_by_heading split on ## and nothing else, so a document with no ## headings became a single chunk — the largest was 73KB against Cloud's 16KB per-document limit. It now sub-splits on ### , then paragraph breaks, then a UTF-8-safe hard cut.

  • Chunk IDs embed the full source path and two exceeded the 128-byte ID limit. Overflow IDs now carry a path hash; the other 10,177 keep their readable form, because a readable ID is what makes a raw collection dump diagnosable.

Graphiti. The changelog knowledge graph is ingesting into Neo4j — 522 dated entries as bi-temporal episodes. It runs at roughly one entry every 25 seconds and persists state after each success, so it resumes from wherever it stops.

What's next

  • Finish the Graphiti pass and query the graph for what the changelog tier can answer that vector search cannot.

  • The sweep surfaced 22 documents whose sections blew the size cap. They are the corpus telling us which documents have outgrown themselves, and are the natural candidates for the fork-and-cross-reference discipline.

  • Two artifacts in the corpus are not documents at all — a generated wikilink path audit and a hash-named scrape cache. They probably should not be indexed.

  • [[Sweep-for-Frontmatter-Consistency-&-Improvements]] — the method this ran

  • [[Nine-Hand-Rolled-Frontmatter-Parsers-All-Functionally-Identical]]