Sitemap, robots.txt, and an OG card — the rest of the discovery…

Why Care?

A splash page that no one can find is a splash page that doesn't exist. The [[2026-05-09_02|llms.txt shipment from Saturday]] solved discovery for one audience — agentic crawlers and LLM ingest tools that follow the /llms.txt convention. But there are two other audiences who also need to find this site, and they speak older, more boring protocols: Google et al. want sitemap.xml plus robots.txt, and the unfurl bots behind iMessage / WhatsApp / Slack / Discord / X want a properly-sized OG image bytes-matched to its declared MIME type.

None of this is interesting in isolation — it's solved-problem territory from 2010. What's interesting is that all three protocols now point at the same corpus from the same splash, and every page in the kit is now reachable in three different machine-readable shapes (an XML index for crawlers, a markdown index for LLMs, an HTML page for humans). That's the whole "context vigilance" thesis applied to the splash's own discoverability.

What's New?

@astrojs/sitemap integration in splash/astro.config.mjs — auto-generates sitemap-index.xml + sitemap-0.xml from every page Astro emits. A filter callback excludes the /llms.txt and /llms-full.txt endpoints (those are for LLMs, not search engines) and the 404 page.
splash/public/robots.txt — three lines: User-agent: *, Allow: /, and a Sitemap: pointer at the absolute URL of sitemap-index.xml. No bot is denied; the sitemap line is the load-bearing part.
Two discovery <link> tags in BaseLayout.astro — <link rel="sitemap" type="application/xml" href="…"> so crawlers that look in <head> find the sitemap without parsing robots, and <link rel="alternate" type="text/markdown" href="…llms.txt"> so any tool that treats rel="alternate" as an ingest hint can find the markdown corpus.
A 1024×1024 JPG OG card — splash/public/ogimage_Context-Vigilance_1024x1024.jpg, 977 KB. JPEG, not WebP. The choice is deliberate (see below).

How the three protocols compose

                      ┌──────────────┐
                      │   splash     │
                      │   /          │
                      └──────┬───────┘
                             │
       ┌─────────────────────┼─────────────────────┐
       │                     │                     │
       ▼                     ▼                     ▼
  /sitemap-index.xml     /llms.txt            /<og:image>
  /robots.txt            /llms-full.txt       (1024×1024 JPG)
       │                     │                     │
       ▼                     ▼                     ▼
   search engines       LLM crawlers          unfurl bots
   (Google, Bing)       (ingest tools)        (iMessage, Slack)

Three protocols, three audiences, one corpus. The same publish !== false && private !== true gate governs all three — if a page doesn't show up for humans, it doesn't show up for any of them.

The OG image bit — why JPEG and not WebP

The instinct on a 1024×1024 share card in 2026 is to reach for WebP — smaller files, modern format. Don't. The share-preview bots behind WhatsApp and iMessage are not modern browsers — many of them choke on WebP, and worse, some image CDNs do content-negotiation that hands a WebP body back even when the URL ends .jpg, leaving og:image:type lying about its bytes. The unfurl silently fails and you don't notice because the link still works for humans.

Encoded JPEG, the byte-stream's first few bytes literally start with FF D8 FF, the declared og:image:type says image/jpeg, the URL ends .jpg, and the file is hosted out of splash/public/ so no CDN gets between us and the bots. Three points of agreement, zero negotiation. The card unfurls.

A spec-conformance gap (same one llms.txt has)

Both robots.txt and the sitemap-discovery convention assume the file lives at the host root: https://contextvigilance.com/robots.txt, https://contextvigilance.com/sitemap-index.xml. Until DNS for contextvigilance.com flips, this splash deploys under a path: https://lossless-group.github.io/context-vigilance-kit/robots.txt.

That means convention-based discovery (a crawler that just GETs /robots.txt from the host) won't find this splash's file — it'll hit lossless-group.github.io/robots.txt, which is GitHub's own. Crawlers pointed explicitly at the path-based URL via a sitemap submission or a rel="sitemap" link in <head> still work today. This is the same gap [[2026-05-09_02#A spec-conformance gap (resolved when DNS lands)|the llms.txt entry called out]], and it has the same fix: the custom-domain rollout already on the roadmap.

Verification

$ pnpm build
... 464 page(s) built in 8.12s
$ ls -lh dist/sitemap*.xml dist/robots.txt dist/ogimage*.jpg
.rw-r--r--@ 106  dist/robots.txt
.rw-r--r--@ 64k  dist/sitemap-0.xml
.rw-r--r--@ 235  dist/sitemap-index.xml
.rw-r--r--@ 977k dist/ogimage_Context-Vigilance_1024x1024.jpg

After deploy, the unfurl test is the honest one: paste the splash URL into iMessage, WhatsApp, Slack, Discord, and X. The card should render in all five with the new image. If any one fails, the bytes / MIME / URL agreement broke somewhere — most likely a cache; cache-bust by querying ?v=2 and try again.

What's next

The sibling memopop-ai splash has the same uncommitted changes ready to land (same astro-sitemap config, same robots.txt pattern, same BaseLayout links, plus its own llms.txt scaffolding). After that, the same trio rolls out to astro-knots/splash, content-farm/splash, and lfm/splash — each gets its own OG card generated against its own DESIGN.md imagery recipe, but the sitemap + robots.txt + <link> wiring is mechanical.

[[2026-05-09_02|llms.txt + llms-full.txt — give the model the corpus directly]] — the LLM-facing half of the same discovery surface, two days earlier.
The sitemap protocol
The robots.txt spec
The open-graph-share-seo-geo skill in lossless-skills/ — encodes the JPEG-over-WebP rule, the og:image:type-must-match-bytes invariant, and the cache-bust recipe.