[Red5d]

(Developed in collaboration with my Hermes agent. Reviewed and verified working in a local docker container by me, the human)

Implements the llmstxt.org v2 proposal so LLM agents can navigate the site:

File	Purpose
/llms.txt	4 KB site map: shows (with episode counts), core pages, per-show index links
/show/<slug>/llms.txt × 8	Full episode indexes (1,961 episodes): date · title (guest) - description, newest-first, each linking its markdown twin
/llms-full.txt	Core (non-episode) pages concatenated (~124 KB). Episodes deliberately excluded — the full corpus is ~15 MB, a bit large in a context window
<url>index.md twins	Markdown version of every page/section (spec form for extensionless URLs). Episode twins include show notes, hosts, guests, duration, audio/video links

HTML pages get spec discovery links: rel="describedby" → /llms.txt, rel="alternate" type="text/markdown" → the page twin.

How it stays current

No scheduled job, no committed generated files. scripts/gen_llms.py runs in the Docker builder stage on every build and regenerates everything from Hugo's own rendered data (an NDJSON dump emitted during hugo --gc, then deleted). Deploys already fire on every push to main, including scraper auto-commits, so new episodes land in the indexes automatically. Drift is structurally impossible.

Notes for review

• Dockerfile gained a small llms stage (python:3.12-alpine) because the pinned hugo image's apk repos resolve to current Alpine and its python3 fails to load against the image's musl. Final stage now copies public/ from llms instead of builder.
• nginx: stock nginx:alpine has no text/markdown mime mapping, so a location ~ \.md$ block sets the content type. The inserted blocks repeat root because the stock conf declares it inside location /.
• Episode URLs: indexes use Hugo's real permalinks (/show/linux-unplugged/650/, numeric slug), not content filenames (0650.md).
• config.toml: [outputs] home now explicitly lists ["html", "rss", "json", "llmsdump"] (previously Hugo defaults) — no behavior change for existing formats.

Verification (full docker build of this branch + served-container checks)

• /llms.txt, /llms-full.txt, all 8 /show/<slug>/llms.txt: 200, spec-conformant structure (H1 → blockquote → H2 file lists)
• /show/linux-unplugged/650/index.md, /membership/index.md, /live/index.md: 200 text/markdown
• <link rel="describedby" href="/llms.txt"> + markdown alternate present on pages
• Unit tests: test/test_llms_gen.py 4/4; e2e checks added in test/e2e/test_llms.py

[CGBassPlayer]

Thanks for the contribution! I am currently just getting back from being out of town. I will look this over as soon as I can

[CGBassPlayer]

So I was looking through and I think it would make more sense for this to be done with Hugo itself rather than a python script during the docker build. You should be able to use custom output formats and templates similar to what you are doing.

• /llms.txt and /llms-full.txt: custom text/plain formats on home, with layouts index.llms.txt and index.llmsfull.txt.
• /show/<slug>/llms.txt and episodes.json: formats on the show sections, with show/list.llms.txt. Enable them with outputs = [...] in each show's _index.md front matter, so they don't also render for /show/, /hosts/ and /guests/.
• Episode lists: (site.GetPage "/show/<slug>").Pages.ByDate.Reverse, with first 10 for the recent list.
• URLs: use .Permalink instead of the hardcoded SITE_BASE.

Going this route:

• removes the need of the script and cleanup steps in the build
• it allows the llm text files to work in our dev instance on render as well as locally
• it makes it so all episodes work. Your current implementation will break on the launch's "T" episodes and office hours pocket episodes as they are prefixed with "T" and "p" respectively. It will also break if (and when) LUP hits episode 1000 since you prefix is only looking for a leading 0.

I do love the idea of making the site more LLM friendly. Let me know if you have any questions.

[Red5d]

Sure, makes sense. I've pushed some updates to the PR to implement it with just Hugo as you suggest, and handle the "T", "p", and 1000+ numbered episodes.