Nine Keyboard Widgets That Were Never Implemented
Seven members promised a screen reader arrow-key navigation and none delivered it. Closing that took six more primitives — and the first tests this federation has ever had.
lossless-group/augment-itrebuild/turbo-rsbuild
1,127 commits · +240,339 lines
Seven members promised a screen reader arrow-key navigation and none delivered it. Closing that took six more primitives — and the first tests this federation has ever had.
Forty deployable units, fourteen READMEs, zero component design systems. We audited our own flagship against the advice we give clients.
The graph moved 23 nodes in five weeks. Underneath, a quarter of it was replaced — including a fix we proved, then stopped applying.
Ten byte-identical service configs folded onto a shared base — and the audit that proved it found three defects nobody was looking for.
Nobody had ever seen the eleven status pills next to each other. Put on one canvas, the promotion argument makes itself in about four seconds.
Nineteen units, 302 raw buttons, 824 net lines of CSS deleted. The buttons were the easy part — the loop got four revisions from its own engineers' reports and is now the thing worth keeping.
Twelve units in one evening, deviations from eight to one, and the gallery caught a contrast defect in the page that documents contrast.
The third primitive reached every member in a night. Three engineers independently proved one of them was keyboard-dead, then came back and adopted it once it worked.
Three corpora showed identical sources under three names. The data was right; the UI was answering a question it had stopped asking.
Production was serving a build from August 3rd. Every deploy since had failed at the same line, in all six frontend Dockerfiles, for the same reason — and nothing was watching.
A client's research corpora vanished from the workspace. The data was fine, the database was fine, the service was healthy and logging nothing. Two counters found it: a single unanswerable message had taken the whole request channel down until restart.
Thirty-five design tokens had been verified by arithmetic and never once rendered. We built the page that renders them, and it found three bugs in itself and three in the theme before it had been open a minute.
The token layer got a surface last week. The other half of the design system — seventeen members' own components — still had none. Now each member ships its own component library, at its own address, with a live contract audit attached.
The surface has said 'Corpora Curator' on screen since early July. Underneath, every file, class and federation remote still said strategy — a word that was only ever one value out of five. The code now says what the thing is.
'No corpora yet' showed up twice in one week from two unrelated causes. The first was a dead NATS subject. The second was one guessed word, filtering a list nobody ever asked to have filtered.
A knowledge graph over the codebase found eleven micro-frontend mount files differing by a single line each, 250 lines nothing imports, and a task runner that was never installed. All of it is gone — 466 lines lighter, 19 packages building, 1,494 files typechecking clean.
Ask a hard question about a micro-frontend architecture and you sometimes learn the answer is 'it renders invisible text and tells you everything is fine.' We measured that failure in a browser, then gave every design token a floor so it can't happen.
Two operator-reported curator bugs, fixed the same afternoon they surfaced: comma-separated tag entries now become separate chips instead of one fused mega-tag, and 'Fetch full content' no longer overwrites the title (and bib) you typed — it fills only empty fields and, when a page can't be read, says so instead of silently resetting the title to the URL. Plus the search-queue feature earns a splash highlight.
The metadata parser was reading only blog/OpenGraph keys, so scholarly articles — whose data hides under Highwire/Dublin-Core/PRISM tags — arrived with no date and a weak publisher, and the operator hand-filled them. Now it detects the source KIND (academic-paper / article / company-landing / web-page), routes to the right parser profile, and resolves every field across all conventions. A Springer article now fills its authors, publisher, and date automatically.
augment-it goes from zero automated tests to 43 across the whole corpora-builder chain — identity, tenancy, transport, canonical CRUD, corpus files, curator state, chat, alignment, and a full backend-chain integration test — spanning two repos and two languages. On the way it caught a real reconnect bug and two live data drifts that no human had flagged.
augment-it gets its first automated tests: Vitest wired, nine tests green across the transport, the workspace registry, and didi's corpora slab. The headline property test — 'no invoke silently vanishes' — failed first exactly as the registry predicted, and the failure was real: the client reconnect chain died on any refused connection, a live mechanism behind the zombie-session and lost-corpus symptoms. Found by the test, fixed in the transport, now guarded forever.
One domain, two client orgs: workspace-service learns the org↔workspace mapping the identity spec always designed, every capability frame gets server-side tenant enforcement, and reach-edu's Stephenie Tesoro becomes the first client user who isn't us.
Org→org edges land in the canonical layer — parent, child, or peer, with a typed flavor and free-text human context — and organizations get their first tags (Initiative, Program, Funder), all workable from the Org Workbench card.
A misclicked ➕ put alpha.school on Princeton and the only undo was a raw database write. Now every list entry and identity property on the org and person cards carries hover-revealed ✎ and × — patch it, or remove it with a one-line confirm.
The search-results rail shipped at dinner and got used in anger by ten. Every rough edge the operator hit — dismissal, concurrency, overflow, a broken drag, an open API-credit tab — was fixed and merged before midnight.
The first real workbench session filed seven issues; the three in today's scope ship together: person cards list their corpus entries instead of a bare count, pulse streams gain a user-facing name and an in-place kind/name editor, and a bio page on another org's site can be promoted to an affiliation in three clicks.
The fourth fix of the day: AdditiveList's fallback display becomes host+path instead of hostname alone, so a blog_index stream at gatesfoundation.org/<path> stops masquerading as the root domain and same-domain entries stop being identical twins. With it, all four issues filed from today's workbench sessions are fixed, shipped, and closed — same day.
The Gates Foundation's Deputy Director rendered as a bare UUID: her row was born from the crawlbase/event-CSV import scripts, which wrote full_name and never name — the field every person read selects. All person reads now coalesce name ?? full_name, fixing 136 of 1232 persons at once; the operator-run backfill makes it durable on data.
The workbench could only find orgs that already exist. '+ New organization' opens a gated create: scored candidates from every match signal surface first, picking one just opens it, and creating is an explicit choice past the gate — with the new org's domain seeded so it's matchable from birth.
The workbench gets its missing first move: a sticky left column listing every org the workspace client can see, filterable by name, sorted by corpus count ascending — so the organizations that could and should have more corpus content surface at the top, zero-corpus in red, one click from their card.
The workbench's 🤖 era begins: one organization.crawl capability (Sonnet + Anthropic server-side web search) crawls for relevant identity links, pulse streams, or team members, steered by a per-workspace operator-editable relevance brief — proven live against Aspen in 65 seconds with the brief visibly shaping what came back.
Agent searches run minutes; the operator no longer does. Every 🤖 now returns instantly, the search lands as a card in a persistent right rail, and the card signals when candidates arrive — expand, accept, dismiss.
The service floor under the new Augment-from-DB flow is in and proven over raw NATS: the org card and people-reveal reads, a registry-resolved generic search fire (SearXNG free default, Exa as the newest paid peer), and alias/domain matching in org autocomplete — all before a single pixel of the two new microfrontends exists.
The first of the flow's two new microfrontends: pick 'Augment from DB' at the front door, autocomplete to a canonical org (names, aliases, or domains), and work its card — identity/social links, pulse streams, corpus items — each list additive in place. Plus the verb the spec missed: organization.streams.add, proven live against The Aspen Institute's blog.
The flow's second microfrontend: every 🔍 on the org card opens a paired search rail with the term in an always-editable bar, providers a chip away (SearXNG preselected, Exa/Tavily/SerpApi as peers), and every result row one ➕ from landing on the launching entity's list — with the card refetching the instant it does.
Spec steps 6 and 7 become chrome: every person RELATEd to an org listed with role and relevance, expandable to nested identity links with their own ➕ and 🔍, and an inline add-person where the operator only resolves the human — the affiliation edge and its observation materialize from person.affiliate with the org pre-bound.
Spec step 8 ships as a mode of search-and-add: a per-stream Scan button on the org card fires the official-blog machinery with the stream URL as the authoritative index, dedups against content_items, and badges what the corpus already holds — proven live against The Aspen Institute's blog, flag-flip and all. All five phases of the spec are now implemented.
Steps 9 and 10 of the humain-vc unlock build order: augment-it's single-tenant instance is live at augment.didi.sh, on Railway instead of the originally-prepped DigitalOcean droplet — a platform switch made after the droplet's real numbers didn't match the plan, and validated against Railway's actual multi-service project model.
Step 6 of the humain-vc unlock build order: Aniel adds a link, Michael's screen updates without a refresh, and vice versa. The resolver now broadcasts after every domain/source mutation commits; the curator surface refetches on the ones that touch its active view.
Step 7 of the humain-vc unlock build order: a single-tenant deploy now shows a real sign-in wall instead of a half-broken shell to anyone without a didi.sh session. Building it surfaced a genuine architecture gap — an anonymous WS upgrade against a required instance is rejected before any session frame ships — closed with a small, unauthenticated config endpoint.
Step 8 of the humain-vc unlock build order: the chat rail is now didi, it can file a captured link under an existing thesis by name in one turn, and it has an actual face — a character headshot composited with the shell's own brand gradient. Deliberately scoped to augment-it alone; the cross-service didi vision has its own exploration doc and its own timeline.
Build-Order step 5, finished: humain-vc now defaults to 'thesis' on its own (no more typing it every session), and a new domain.retype capability moved the one pre-existing domain — consumer-immunology — from strategy to thesis, DB and filesystem both. Riding along: a real back button in Corpora Curator's header, and a directory-existence bug caught mid-migration and fixed before it shipped broken.
The 'Flows' popdown moved from the header's right side to sit where the numbered bubble strip lives, and now actually switches which flow is active — 'Improve a CSV' (the original 6-step pipeline, with strategyCurator un-spliced back out of it), 'Build Corpora' (1 step), and a third, 'Augment a CSV of Event Attendees' (2 steps), added minutes later at zero architecture cost. FlowWidget's bubble count is no longer a fixed constant; it resizes to whatever the active flow's rotation actually is.
record-db-resolver only ever knew organizations — feeding it a CSV of FreedomFest speakers wrote a person into the organizations table and minted a nonsense opportunity. The fix is a new remote, person-db-resolver: match-or-create a person, then independently match-or-create their org and RELATE the affiliation with a role. Person and org are separate decisions the operator can make in either order, together or alone — not a coupled 1:1.
person-db-resolver shipped a few hours ago wired end to end — but the name field couldn't be typed into, and the observation form refused to save unless you filled in a predicate nobody was prompted for. Both are fixed. Improve a CSV of People is now basic-functioning: pick a record, edit the name, match or create the person, match or create the org, drop a note.
Every check on the canonical layer so far meant writing a disposable Node script, running it once, deleting it. Now there's an MCP server wired straight into SurrealDB Cloud, and a skill that carries the schema knowledge and the verification discipline. First real run: tonight's FreedomFest 2026 batch is tagged correctly across all 65 people and 40 orgs — but only 1 of those 65 people actually got RELATEd to an organization.
The first augment-it flow that starts from SurrealDB instead of a CSV has its rating half working — export the FreedomFest 2026 affiliations as a spreadsheet, rate relevance offline, reimport, and write the rating back onto the affiliations edge. Getting from spec to a working screen surfaced two real architecture gaps (person-enrichment's worklist was coupled to a different person shape than it looked; the rating resolver needed its own escape hatch when an operator lands on the wrong record set) and a lesson about not surgically restarting one process in a --parallel dev stack. More to go — the link/corpus editing half isn't in this screen yet.
The v0.1.0.0 rating loop shipped tonight, then got real feedback the moment it was actually used: two screens for one task was the wrong shape. Reversed course — apps/affiliation-rating-resolver now does what record-db-resolver and person-db-resolver already do for their entities: view the record, edit it in place. Five new capabilities later, the operator worked through 9 real FreedomFest 2026 affiliations end to end — real relevance ratings, real notes, real canonical links, real corpus content — and every byte of it verified correct directly against SurrealDB.
augment-it becomes the didi.sh identity service's first consumer: the workspace WS upgrade now verifies the didi_session cookie locally (jose + JWKS, EdDSA-only), attaches didi_id to the session, and keeps the legacy continuity token working — proven end-to-end against local dev, id service to Docker container.
The header gains a didi.sh identity badge that carries across every micro-app: server-verified didi_id state straight from the workspace session frame, and a sign-in popover that completes the whole magic-link flow without leaving augment-it — the headless GTM contract, working in dev.
The verified didi.sh identity now rides beside every capability invoke into the domain services, stamping created_by/updated_by on domains, sources, and source_usages rows in SurrealDB and created_by into the corpus frontmatter content-ingest writes — no consumers yet, just the wire proven end to end against local dev.
A header-level dropdown (the Lossless 'jumbo popdown' convention, ported from astro-knots into the shell's Svelte header) replaces the memorized-:3017-port habit with a click: 'Flows' -> 'Build Corpora' navigates straight to strategy-curator, full-screen, no CSV-augmentation steps trailing it. No new persisted state, no auth interaction — a plain navigate action, same mechanism the chat rail and pack-runner already use to reach non-rotation remotes.
Strategy Curator is now Corpora Curator, on-screen: the app stopped being strategy-specific the day humain-vc needed the identical shape for theses. Two fixes riding along — the curator was silently stuck on a stale workspace after switching from the shell's header, and the domain type it writes into (previously a hardcoded 'strategy' constant) is now a plain text field the operator sets per corpus, no code change required to start a 'thesis' or anything else.
All 10 services swapped the deprecated monolithic nats package for the scoped @nats-io/* v3 client — 27 files, one codec rewrite, zero wire-protocol changes — and every service round-tripped live on the rebuilt stack before the branch moved on.
Pick a strategy, gather sources for it, and fully control each one: fetch full content, retry past anti-bot pages, edit the bibliographic fields, rename the file, attach a PDF you downloaded yourself, tag it — all writing straight to the corpus filesystem and a SurrealDB registry that never decouples. The landing-page-vs-PDF problem finally has an answer.
Fetch a source and it now fills in its own author(s), publisher, and publication date, straight from Jina's structured metadata — authors as an array, all editable, all written to both the file and the registry. Plus a green save-pulse on every field, and a fix so renamed files actually stay connected.
augment-it grew in two eras that never met: the row-store records and the SurrealDB canonical orgs. A new per-record surface bridges them — match each record to a canonical organization (or create one) and land its web presence as additive, deduped facts. First iteration works out of the box; a little UI polish before it ships.
The resolver now stamps each match straight back onto the source record (so it survives a crash and exports back to the client's CSV), and lets the operator rename the canonical org's name/slug without ever breaking the bond — the immutable id holds, old slugs become aliases.
Confirmed working in-app: you can now correct a canonical organization's name and slug after research, and the match to the original record holds. The bond is the immutable id; the slug is just its editable face, and every match rides back onto the source row.
Resolving a record now mints an opportunity: a first-class, client-scoped entity 1:1 with the source record and many:1 to the canonical org. Three pipeline rows for Accelerate the Future become three opportunities on one org — none merged, none lost. This is also where the client's CRM data finally lives.
528 markdown files on a laptop and 129 organizations in the cloud database knew nothing about each other. Tonight a reconcile pass joins them: every fetched URL becomes a content_items row that records who published it, who it's about, and (soon) everyone it mentions — 500 rows, zero leakage, idempotent and re-runnable.
A VC client runs their fund + CRM on Decile Hub. Tonight augment-it learns to talk to it — a skill that codifies the contract and a TypeScript MCP server that exposes it as agent tools, both grounded in Decile's own OpenAPI spec rather than guesswork. It's the first connector in the per-client seam we've been building toward.
The relationship between people and organizations is a lie at the data model level. How careers work: people sit on boards, advise, change jobs, keep a past CFO title. Tonight the single org slot becomes an open-ended stack of affiliation cards — each with its own free-text role written straight onto the graph edge, each able to find-or-create its org. Start typing a name and existing orgs autocomplete from canonical (so you pick 'Institute for Humane Studies' once, not retype it on every attendee).
Last night the canonical layer landed in SurrealDB; tonight the operator-facing surface that turns sparse rows into rich ones starts running in the browser. The new `apps/person-enrichment/` remote is a *pulse-surface* — a per-event UI where the operator iterates one attendee at a time and fills as many dimensions as one Google search surfaces: name (paste a full name and it splits surname + first_name; or edit either field directly), additional emails, personal links (LinkedIn, X, Substack, GitHub — kind auto-inferred from URL), personal corpus (the LLM/RAG ingest target — articles, podcasts, interviews), an organization (find-or-create by slug), org links, and org corpus. Every input commits on Enter; every save announces the doc-table AND relational-table writes it caused; corpus URLs get a shared UUID across the `content_items` relational row and any number of `personal_corpus` / `org_corpus` array entries, with in-browser verification confirming the cross-doc id matches; orgs auto-detect on the next attendee from the same domain so the operator stops typing 'Stand Together Foundation' fourteen times. Two new context-v docs — the spec naming the pulse pattern, the issue file capturing the query lenses we'll want once `has_personal_link` observations accumulate. 14 of 177 reach-edu attendees enriched in real use this evening before the operator paused for a safety commit.
A real consulting engagement walked in asking for a curated NYC dinner list, and the org-first frame augment-it has been living in didn't fit. Tonight: ~250 LinkedIn profiles captured via Crawlbase land as flat CSV + full-shape JSONL (skeletons flagged for retry, no fields silently dropped), ~200 LinkedIn 'Save to PDF' downloads get slug-renamed via pdftotext extraction, a browser snippet writes a forward manifest so we never need to reconcile by hand again, the deep-profile DOM extractor catches up to LinkedIn's late-2025 hashed-class layout (pronouns / followers / connections / website / cover photo / robust headline classifier), and three composition scripts join everything into a tag-grouped markdown briefing and a brand-themed HTML export. The exploration sitting alongside the code names what just happened structurally: org-first and people-first are two pivots over the same org↔people join, and a canonical layer (the LinkedIn pipeline) belongs cleanly separated from a proprietary layer (per-engagement commentary). Sub-scale (~30K enumerable venture entities) is the differentiator the big CRMs structurally can't occupy.
Yesterday's [[Joined-People-UI-and-the-Network-First-Pivot]] exploration ended with a clear conclusion: the filesystem-as-substrate posture that has carried augment-it this far stops working the moment we try to blend canonical entity data (LinkedIn truth, refreshable, shareable across clients) with proprietary per-engagement commentary. Tonight that posture changes. SurrealDB Cloud is live on `main/main`, wired through the Node SDK with a working connector, seeded with the 882 humain-vc LinkedIn-network persons from last night's tagged-briefing pipeline AND the 177 reach-edu attendees from a Stand Together event held last month — two clients sharing one canonical schema with not a single row of leakage between them, courtesy of a `client_access` array materialized on every entity. Plus the spec for the UI that closes the loop — a per-event enrichment surface where the operator turns 177 email-only sparse rows into named persons + properly-tagged organizations one row at a time. The fact log (`observations` table) captures every claim with provenance — who said what, when, on behalf of which client — so the next time we refresh from LinkedIn or pull a new attendee list, nothing overwrites an operator's hand-curated note. The bus that left yesterday with a network-first pivot is back tonight with the data substrate to make the pivot real.
Tonight started with a small symptom — three rows in the Sort & Filter Lens on the v10 record set showing `corpus 0` despite per-funder corpus directories full of files on disk: hewlett-foundation (2 files), howard-schultz-foundation (6), kellogg-foundation (6). The yesterday-night issue file had named four others (sobrato, stand-together, cohen, todd-fisher) and proposed a per-client `corpus-overrides.yaml` as the defense-in-depth fix. Hard-refresh resolved those four cleanly — but exposed these three. A second NATS probe confirmed the backend was returning the correct counts (2 / 6 / 6) end-to-end for the new symptoms too. The actual bug was in the wire, not the join: `corpus.list_for_record` had no `CAPABILITY_TIMEOUTS_MS` entry so dispatch fell through to the 5000ms default, the content-ingest handler is a serial `for await` loop, and each call walked every funder directory under `clients/<id>/corpus/` (~40 dirs × ~300 .md files). With 96 visible rows firing 96 parallel requests on view load, late ones in the burst timed out and the lens swallowed the error silently — race timing explained why sobrato et al. recovered post-refresh and hewlett/schultz/kellogg didn't. The operator's framing: *'shouldn't there be a pretend join where the column in the records sheet for corpus references a folder name/relative path?'* The records sheet already populates `corpus_funder_slug` per row. Wiring that into `listForRecord` as the primary join key dissolved THREE things at once: the bug (operator's explicit assertion routes around every lineage edge case), the timeout race (per-request walk went from 300 files to ≤13), and the planned `corpus-overrides.yaml` proposal (the cell IS the override surface — no new file format needed). Lineage stays as the fallback for rows without a slug. While the door was open we noticed something else: Jina's response body has a `Published Time:` preamble line on every capture, and we'd been writing it to the body but never lifting it into frontmatter. The sort/filter UIs had no first-class authored-date field. A six-line addition to `jina.ts` parses the preamble + a `liftPublishedAt` helper in `corpus.ts` promotes it between `fetched_at:` and `record_id:` on both writers. A sibling backfill script rescued 244 dates from existing files (some going back to Wikipedia article creation dates from 2004 — the corpus carries deep history we couldn't see). Also tonight: 226 hand-curated captures across 20 funder directories + 86 inbox triages from the operator's v10 hand-search rhythm, including the first `manual-local-pdf` convention (operator drops a ResearchGate PDF from a local download; sidecar .md flags the missing canonical URL for later replacement). The slug-join means each of those 20 new funder dirs surfaces on its row's chip the moment the dir exists, no record_uuid plumbing required.
augment-it was filing all its work under one tenant (reach-edu) and the slab that told the chat what `client_id` to use literally hardcoded the string. Tonight that ends. The shell grows a workspace switcher in the top-right of the header that lists every directory found under `clients/` (today: humain-vc, reach-edu), and clicking one persists the choice to localStorage, broadcasts it via `augment-it:workspace-changed`, and tells the workspace-service which `client_id` is process-wide active. The chat slab now reads the operator's pick from `ctx.client_id` (forwarded on every chat turn) and falls back to the server's process-wide active value — the hardcoded reach-edu string is gone. The load-bearing piece is the env-var seam: `clients/<slug>/.env` gets loaded into a frozen, in-memory map keyed by slug at workspace-service boot. We deliberately do NOT merge into `process.env` because the workspace-service runs in one process and serves all tenants — `process.env` would bleed Decile keys from humain-vc into a reach-edu chat turn. The connector-config seam (LLM / search / CRM / MCP / storage) the spec calls for lands in step 2; tonight surfaces the raw env, hides nothing, and gives Decile a clean home to slot into next.
The chat surface (`apps/chat/`) has been mechanically functional for weeks — Anthropic SDK in `services/prompt-runner/src/chat-turn.ts`, four-cache-eligible system slabs assembled in `services/workspace/src/chat.ts`, three-mode tool dispatch (chat_answer / chat_propose / chat_invoke), Svelte transcript with accept-proposal cards — but it knew about three verbs (`prompt.draft`, `prompt.improve`, `prompt.apply`) and one context dimension (`record_set_id`). For the funder content corpus workflow the operator is actually in, that meant the chat had nothing to offer. This commit ships the first non-prompt verb: `/inbox <url>` recognized by the model, dispatched as `corpus.inbox.add`, writes to `clients/reach-edu/corpus/inbox/` with the extended frontmatter contract from the Corpus-Inbox-Capture-and-Triage spec. Claude is the default and was the default; what changed is the chat now has something useful to do with the API key it already had. Companion spec `Chat-Context-Awareness-Architecture.md` lands alongside, naming the three v0.0.1 design holes the inbox verb exposed (hand-written verb roster, hardcoded active_client_id in the context slab, ActiveView covers only half the microfrontends) and defining a three-layer migration plan (workspace as context broker → slab-assembly contract → verb registry) that each layer ships independently. Path B per the user's pick — ship the verb tonight, spec the architecture once it's working.
Three things shipped in one short session against the live :3002 surface. First, the per-record field renderer in Record Collector stopped silently hiding structured values — arrays and objects now serialize through the same path as the CSV export (`logic/format.ts`, mirroring `download.ts:csvEscape`), and any empty value (scalar `''`, `null`, `undefined`, `[]`, `{}`) renders a muted italic `(empty)` placeholder so the operator can confirm the prior enrichment pass populated the field before kicking off the next one. The 11px muted JSON style bumped to a readable 12px, normal weight, on the field background. Second, the single `Augment This Set →` button became a small `Augment this Set` header with two stacked CTAs — `Run a Prompt →` (navigates to `promptTemplateManager`) and `Run a Bundle / Packs →` (navigates to `packRunner`) — so the user picks the divergence at the Record Collector surface instead of landing on a default and toggling in-slot. The set-header also restacked vertically so long versioned filenames (`2026-06-05_Master-Pipeline-Tracker--Active-Pipeline_v8.csv`) wrap above the CTA panel instead of squeezing it offscreen. Third, when the operator surfaced that the sidebar treats five sequentially-uploaded variants of the same external tracker (`…_v4.csv` through `…_v8.csv`) as five unrelated peers, that produced `context-v/specs/Record-Set-Family-Grouping.md` (v0.0.0.1, Draft) — a spec that separates internal-lineage families (already in `promoted_from`) from external-variant families (new `variant_family_id` field), names five new workspace capabilities, defines the suggestion-only filename-stem heuristic, and locks that the Augment-this-Set CTAs target the active leaf, never the family.
The spec dropped earlier today (`context-v/specs/Record-Set-Family-Grouping.md`, v0.0.0.1) implemented end-to-end against the live :3002 surface. RecordSet picked up the two new fields (`variant_family_id`, `variant_family_label`); the row-store grew a top-level `variant_families` slot plus five mutators (create / update / add / remove / dissolve) and one read-only heuristic (`suggest_variant_family`) that normalizes a filename stem — strips date prefix, `_vN` suffix, extension — and matches against existing sets with the same stem and ≤ 3-column schema delta. The workspace WS bridge registered the six new capabilities. Browser-side: `apps/record-collector/src/logic/family.ts` walks every leaf back through `promoted_from` to assemble lineage chains, groups leaves by `variant_family_id`, sorts every list — groups, members, archived ancestors — newest-on-top per the user's mid-implementation correction (spec edited to match: §Sidebar rendering now reads 'Newest-first is uniform across the sidebar'). `RecordSetsList.svelte` renders a collapsible family card with chevron header, member count, optional generation count, two inline action icons (✎ rename, × dissolve), and an 'Earlier generations (N archived)' sub-section under any leaf with promoted predecessors. Default sidebar query excludes archived sets at the top level. `App.svelte` calls `record_set.suggest_variant_family` after every ingest; if a match comes back AND the user hasn't dismissed that stem before (sticky per-stem in localStorage), a non-blocking 'Looks like a variant of X (N existing sets) — link as family?' prompt appears under the ingest status with Link / Dismiss buttons. Backwards-compatible — every new field defaults `undefined`; pre-existing stores deserialize without a migration. Existing `…_v4`–`…_v8` sets render as solo cards until the next ingest fires the suggestion.
Three coupled drops in the same session. First, the URL auto-detector + clickable rendering plan dropped earlier today implemented end-to-end in Record Collector — `apps/record-collector/src/logic/format.ts` grew a `FieldShape` discriminated union (`empty | scalar | scalar_url | url_list | json`) that picks by VALUE-shape never field-name, plus a `UrlEntry` extractor that surfaces `url + chip (from pack_id) + label (from label or display_name)` from `Array<{ url, ... }>` payloads. The structured-value branch in `App.svelte` switches on `shape.kind`: `url_list` renders as a vertical flex of clickable `<a target="_blank" rel="noopener noreferrer">` entries showing hostname+path (truncated at 60 chars with full URL in `title=`) so the operator can verify links resolve without leaving Record Collector. Auxiliary metadata (display_name, confidence, source_metadata, response_id, accepted_at, ...) drops from the rendered view but stays on the row and round-trips through CSV. Read-only by design; the per-entry remove affordance lives in a sibling plan, not this one. The keyed-each was unkeyed mid-implementation when `(entry.url)` was identified as a Svelte 5 duplicate-key crash risk against any row whose URL array has repeats — for read-only lists, positional iteration is the right call. Second, a Response Reviewer drill-through audit on the live response-store revealed 1,982 responses with 1,109 (56%) from the three Entity-Pulse OfficialPulse packs at a combined 3-accept rate — `official-blog-pack` 99.5% reject, `official-social-posts-pack` 100% reject, `official-pressrelease-pack` 100% reject — confirming the operator's 'tons of junk links' read of the surface. Domain breakdown showed 65% of pressrelease responses come from `news.google.com` (the google-news-rss connector with weak entity-name binding); 45% of social-posts responses come from `youtube.com` (weakly bound). The packs with strong identity binding — wikipedia (matches article title to entity), linkedin (returns entity's own page) — reject at the normal 73-82% rate for a triage workflow. Third, the operator's response to the junk-volume reframed the next-feature direction entirely: STOP triaging URLs one-at-a-time in Response Reviewer; START treating those URLs as a seed list for a Jina.ai content-ingestion pipeline that writes deduped markdown per funder; build a per-client content corpus that powers cross-funder fundraising-strategy synthesis and per-funder outreach customization. That direction surfaced the broader ai-labs architectural question the operator named explicitly — when does the single-operator-on-localhost posture stop scaling, what stack do we reach for, and how do we architect today so the move is cheap when it comes? — and produced `context-v/explorations/Per-Client-Privacy-and-the-Path-Off-Local.md` (v0.0.0.1, Draft) mapping five candidate stacks (defer-everything, per-client Railway single-tenant, multi-tenant SaaS shape, hybrid posture, managed BaaS) across five axes (repo topology, storage substrate, identity/auth, multi-tenant data model, sensitivity constraints), naming six decision-forcing functions, and five architecture choices that cost almost nothing today but preserve cheap optionality for every path-flip. Operator signed off on Path D — hybrid posture — and committed to start local: `lossless-group/augment-reach-edu` created as a private repo (default README); `clients/reach-edu` registered as a git submodule pointing at it. The reach-edu corpus + operational data + Jina ingest pipeline land in that submodule in follow-up sessions; the augment-it parent stays on the docker-local posture until one of the named forcing functions actually fires.
An end-of-day shipping note that is not about success. The reach-edu funder content corpus workflow was supposed to land tonight against the 96-row Master-Pipeline-Tracker. Final state: 75 markdown files written across 15 funder subdirectories under `clients/reach-edu/corpus/`. 81 of 96 funder records have zero corpus content. The operator's framing at session-end: 'this is all fucking bullshit, nothing is working the way it should, I was only able to process like 1/3 of the plausible records.' This changelog is the honest version. The goals spec landed (Funder-Content-Corpus-Workflow.md, eight rules + six-step workflow), the implementation rebuild against it landed in code (content-ingest service, Content Reader UI, pack honors operator curation, dispatch refuses invalid URLs, fire_id stamping), and 25 broken-URL rows were force-repaired across 5 promotion generations. But the end-to-end validation — fix URLs, re-fire entity-blog, evaluate fresh data — never happened because the session consumed itself patching display-layer symptoms of yesterday's broken-pack-broken-URL data instead of producing fresh clean data via a re-fire. The honest accounting: substantial code shipped, partial data landed, root causes mostly unfixed, eight named failure modes documented in `context-v/issues/Funder-Corpus-First-Session-Failed-Most-Records-Unprocessable.md` for morning-self to attack as a punch list.
After a long arc that re-spec'd the bundle/pack flow from scratch and ended a stack of architectural overreach, augment-it now has a working per-record surface where the operator picks ONE record, fires ONE connector (Firecrawl scan, Firecrawl + Haiku agent, or SerpApi `site:` search), watches the candidate URLs render inline, picks one (or pastes/edits a custom URL), and moves on. Acceptances land in an array column so an entity with multiple canonical paths (Arthur Blank Foundation's /news/ + /blogs/, Ascendium's /newsroom + /our-stories) carries all of them. A save bar at top AND bottom of the records list invokes the existing `record_set.promote` capability with confirmation, the new version auto-becomes the active set, and a post-promote nav surface offers Enhanced Records (download), Augment (next pass), or stay. The same session also produced three new context-v documents that explain the rearchitecture, audit the v3 → v4 → v5 chain to clear promote of a wrongly-pinned bug, and lock the row-count-stable-across-versions invariant. Record Collector got decomposed into proper components with a descending-by-created_at sort and a per-card CSV download button. The user ran the full loop end-to-end on the 96-row pipeline tracker, accepted 98 URLs across 45 rows, and promoted v5 → v6 cleanly.
First implementation step of the Entity-Pulse-Bundle spec lands as a working two-stage pack (find-index → extract) plus a generic ISO-8601 normalizer that solves date extraction across heterogeneous web sources. Smoke-tested against reach.edu: 5 real blog posts pulled end-to-end with ISO publish dates + computed age_days in 4.4s. SerpApi + Firecrawl wired as new connectors; new NATS subject `pack.entity_pulse.requested`; standalone CLI driver for one-off fires and future Agent Chat integration.
The reframe that took shape in late May lands as code: search-provider choice is now an architectural axis of the stack, not a Tavily-vs-anything-else swap. SearXNG arrives as a self-hosted container peer to Tavily, with its own connector under a new connector registry; the common-seven social packs flip their default to SearXNG (no key needed); Tavily stays wired in as the peer for content-RAG packs that need it; and a `provider_override` parameter threads from the response-reviewer's per-record buttons through workspace and social-search so the user can fire any pack through either provider on any row without touching the pack definition. The pre-flight surface for per-row iteration is now in place — the iteration loop itself still pending.
What started as live demo-prep — patching a sticky Fire button, wiring a dead 'Do another round' affordance, adding a navigation to where results land — became a verdict: augment-it's affordances hide, die, mismatch, and (a newly emerging fourth shape) misname. Rather than patching one more thing, the session pivoted to spec mode. The result is a whole-shell UX coherence audit with eight locked decisions, twelve evidence items, four surfaces audited, and three open architectural questions; four sibling context-v stubs (per-remote in-app API docs, initial user experience, the dual-identity blueprint, the auth-patterns-following-Astro-Knots blueprint); and a substantive pickup doc so the next session can resume cold. No code shipped from the spec arc itself — everything held at the sign-off gate — except three small forward-affordance fixes that landed alongside as a separate fix commit and are recorded in the spec's evidence log.
We thought the spec called for two patches and a rename. Halfway through the second patch the architecture broke in our face — a Split view with a working ✎/⊞ toggle but a stubbornly empty pane next to it — and we realized the toggle didn't belong inside either remote, it belonged to the slot. The fix turned into a shell-level concept that pays off in every layout mode.
Over the last two days the response-reviewer microfrontend at localhost:3005 went from a basic review UI to the triage surface for every pack augment-it will ever run. The structured-output extension that landed on 5/25 was the seam; what followed was a fast sequence of commits — many attempts, two scraps, real-data smokes — that turned the remote into a per-row, per-pack triage cockpit with inline correction (URL), human-supply (display_name), and entity-name cleanup (the row's identity column) all riding the same response.set_structured subject. Two emergent-requirements rounds also got captured into the Packs-and-Bundles blueprint so future packs across the Lossless family inherit the discipline. The SearXNG-substrate decision is filed as an issue but deliberately not yet executed — the remote is good enough that the next session can start somewhere new.
Ten commits between 2026-05-25 17:08 and 2026-05-26 02:53 carry augment-it through a complete arc: a tidy-and-status-sweep across the existing context-v tree, a new exploration (Entity-Profile-Augmentation-Workflow) that surfaced a two-tier abstraction, a forked blueprint (Packs-and-Bundles-Pattern) that locked the contracts, a new shared package (@augment-it/shared-ui with the first reusable component — ConfidencePill), a structured-output extension to the response-store schema, a new federated remote (pack-runner at :3009), a new backend microservice (social-search) with six (then seven) packs and Tavily as the v1 connector, a row-store write-back surface (row.fields.socials), a paired-authoring shell rewire that took pack-runner out of the rotation and made it a peer to prompt-template-manager, a complete by-record triage view in response-reviewer (:3005) with inline URL / display_name / entity-name editing, two design pivots that got documented honestly (the profiles.<source> column scrap, the Run-entity plan deferred to its own arc), and emergent-requirements promoted into the blueprint as a §Triage Surface UX Requirements section for the rest of the Lossless family. The system that exists at the end of these two days is recognizably the same shape as the system at the start and recognizably a different product.
augment-it now has a chat panel that doesn't just execute on your fundraising-pipeline records — it helps you author the prompt that drives the enrichment. You state a goal in your own words; the chat drafts a prompt template against a sample of your record set's columns and renders it inline. You react to the draft conversationally, refining it across a turn or two. When it's right, you click run, and the same chat surface narrates the enrichment job's progress and checks at the end that the columns the prompt promised actually got populated. The structural answer to bulk-AI-enrichment-gone-haywire is conversational gating: draft, improve, then apply explicitly. This entry walks through what shipped, how it works, and what v0.0.2 brings in.
augment-it now has a checkpoint surface between enrichment rounds. You finish a pass of LLM-driven enrichment, open Enhanced Records, see every record in one place with its accumulated columns, and click Promote — the system snapshots the current state into a new canonical record set under a deterministic, versioned name, archives the predecessors, and gives you a clear success message with a one-click path back to the start of the arc to author the next round. record_uuid follows each record across every promotion so a single conceptual record stays traceable from original CSV through every round of refinement. This is the loop the walking skeleton has been building toward: enrich, triage, checkpoint, re-enrich on a cleaner base.
augment-it is a multi-tenant pipeline tool. Different clients will upload spreadsheets with wildly different columns. The code cannot have opinions about specific field names. Today we ripped out every place where it did: no hardcoded reserved-field lists, no field-name-aware special rendering, no schema decisions imposed on tenant data. The cell renderer is type-driven (scalar → text, object/array → JSON), the canonical-schema union is presence-driven (any key with values is a column), the promote-fold is type-driven (arrays merge, objects merge, scalars overwrite). One rule everywhere: whatever a tenant puts in row.fields stays in row.fields and ends up in the promoted record. This entry shows the messy middle — what we got wrong, how the user pushed back twice, and what the corrected discipline reads like.
augment-it is growing two new pipeline stages: a request-reviewer that shows you exactly what is about to be sent to the model, and a response-reviewer that lets you triage what came back. This entry is the service layer beneath both — the no-UI plumbing. Its load-bearing idea is small and strict: there is now exactly one function that assembles a model request, so the request you preview is byte-identical to the request that fires. Alongside it, a new response-store service turns every fired response from a throwaway cell into a first-class object you can inspect and flag. The two frontends that sit on top are their own story — coming next.
The shell's tiling Deck now shows the whole enrichment pipeline. request-reviewer and response-reviewer join record-collector and prompt-template-manager as federated remotes — scaffolded, wired into the shell, connecting to the workspace, and tiling in pipeline order. The panels are deliberately placeholders: this entry is the federation wiring, not the review UIs. But for the first time the Deck reads end to end — collect, author, pre-flight, post-flight — in one window.
This morning, Request Reviewer and Response Reviewer were empty placeholder panels. Tonight they work. Request Reviewer shows the resolved request — the prompt with its {{tokens}} filled from a real record row — lets you pick the model, see the literal JSON body, and fire it. Response Reviewer steps through what came back and lets you flag or accept each response. And the boundary between the two is now strict: the request belongs to Request Reviewer, the response to Response Reviewer, and authoring stays in Prompt Templates — which lost its run button entirely. Four stages, four remotes, one gate each.
Last night's commit shipped Request Reviewer + Response Reviewer as code that typechecked. Tonight they survived a 207-record enrichment run against a real CSV with the user as the human in the loop — and every place the loop was thin, this round repaired. The Save / Apply ergonomics on Prompt Templates, the firing indicator and Cancel button and coverage strip on Request Reviewer, the per-row 90-second timeout and run-level cancellation in prompt-runner, the autosave-on-blur on Response Reviewer's edit textarea (after the first run lost 75 manually-typed URLs), helpful_links as a side-channel field on every row, a needs-human triage flag, an icon action row with instant CSS tooltips, per-bucket counts on the filter chips, a fire-time blank-row filter at csv ingest, and the spec for the next checkpoint — enhanced-records-list and the promotion loop — captured in context-v as the build target for the next session.
Phase 1 of the augment-it rewrite landed today. Three containers come up cleanly with one command: a NATS message bus, a thin Workspace Service that owns no domain data, and a row-store microservice that owns its own rows. The shape of every future service is set; the wire is now visibly a microservices wire instead of point-to-point HTTP. The in-app chat surface has something real to plug into.
Phases 2 through 5 of the walking skeleton landed in a single session. Five containers running, the two real fundraise spreadsheets ingested, cells editable from a Svelte 5 frontend that consumes the workspace singleton, and the architecture demo's strongest claim — 'adding a new service is cheap' — has gone from slide to docker-compose diff. The 27th service made its first appearance as xlsx-ingest, with zero changes to row-store.
augment-it now has a real Module Federation host. The shell at :3000 mounts the record-collector remote at :3002, and the architecture has a federation boundary for the first time. Getting there meant fighting three subtle Svelte-5-plus-MF interactions that the official docs gloss over: shared-singleton factories that wouldn't register, $effect_orphan errors from cross-runtime mounts, and component-scoped CSS that quietly disappears across federation chunk boundaries. Each one has a fix worth keeping.
augment-it now does the thing it is named for. Two new microservices land — prompt-store holds prompt templates, prompt-runner makes the LLM calls — and together they close the augmentation loop: write a prompt with {{column}} placeholders, run it per-row against a record set, get back a derived record set with the model's answers as a new column. The proof-of-life was a web-search-backed prompt that turned a column of organisation names into a column of website URLs. This is the first time augment-it has called an LLM.
Yesterday's question was 'how do I see it?' — the enrichment loop worked but only from a smoke script. Now it has a UI. prompt-template-manager is augment-it's second federated remote: a prompt editor with a live {{token}} strip, a bind check that turns green or red against whichever record set you pick, and a run control that streams progress. It mounts as the second tab in the shell. The five-phase walking skeleton from the plan is complete — you can author a prompt, run it, and watch a derived record set appear, all without touching a terminal.
augment-it had its colours hardcoded as hex scattered across five CSS surfaces. Now it has a theme system: a shared @augment-it/theme package with one theme.css holding a two-tier token architecture and three mode blocks, a mode-switcher, and a sun/moon/star toggle in the shell chrome. Light, dark, and vibrant all work across both federated remotes — switched by one data-mode attribute on a shared <html>. The Astro Knots theme blueprint, adapted for a non-Astro, non-Tailwind, federated app.
augment-it's shell stops taking turns. Where it used to mount one federated frontend at a time and destroy it to show another, it now mounts several at once and tiles them like a desktop window manager. Three layout modes: a peek-deck where the focused app sits at 90% with its neighbours peeking from the edges and hover-expanding, a co-existence split where two apps share the viewport behind a draggable seam, and plain full-width. Layout state persists per-user behind a seam the shared-auth store will back later.
A small Astro site at splash/ that ships to GitHub Pages on push to main, renders the repo's changelog/ and context-v/ alongside curated copy about the six module-federated microfrontends.
Before writing the next line of code in the record-collector submodule, we wrote down what it actually has to do — input shapes, output contract, edge cases, and where it sits in the broader workshop.
A morning of issue-triage turned the next month of augment-it work into five named threads — and named the loose ends honestly.
Five planning docs that had been sitting at the repo root got consolidated into specs/. A small move, but it set the precedent for where future specs would land.
A 3,064-line documentation drop that gave augment-it a written voice — the kind of artifacts a new collaborator can read end-to-end before touching code.
The Card component was duplicated across two microfrontends. Tanuj lifted it into packages/ui as RecordCard, the first non-trivial component to live in the shared package — and the proof point for how cross-MFE components will work going forward.
The bolt.new monolith had served its purpose. We tore the build system down to studs and rebuilt on a turborepo + pnpm + Docker base ready to host federated microfrontends.
Tanuj brought up a minimal-but-real module-federation scaffold — a host shell that loads two remote microfrontends from the same monorepo, each exposing a Card component, all sharing a single ui package.
The two placeholder microfrontends got promoted out of the parent repo and into three named, independently-versioned submodules — the first concrete instance of the federation contract.
Two months after the Bolt monolith hit working-demo state, we tried to peel record-collection off into its own frontend repo. Two days, three commits, one DB swap (turso out, Supabase in), then we set it down to think harder.
A late-January progress update with embedded GIF demos from the pre-restart Augmenter App: API options and editor, response list, user profiles, highlighter, and grouping by customer.
Before there was a federation, there was one big React app: Bolt-vibed, Supabase-backed, three LLM providers wired in, every screen in the workshop already shipping as a component. This is where we found out what the workshop wanted to be.