Curate LLM-friendly documentation indexes with ak:llms

Use ak:llms to create a Markdown index for documentation consumers that follow the llms.txt convention. The bundled generator scans local .md and .mdx files, extracts titles and short descriptions, groups pages heuristically, and writes llms.txt. It can also write an expanded llms-full.txt containing inlined document content, and it defines the per-page .md variant contract that llms.txt can link to when the site serves Markdown twins of its pages.

Choose ak:llms for a curated documentation index

Use ak:llms when

• A documentation site needs a concise, categorized Markdown index with canonical links and descriptions.
• You want a first draft generated from a local docs directory, followed by human curation.
• You need an optional expanded context artifact for a deliberately bounded set of public documents.
• You want a repeatable local generation step that uses only Python's standard library.

Choose another workflow when

• You need framework routing that serves these files, SEO metadata, robots.txt, a sitemap, social metadata, page actions, hosting, or deployment. Those belong to the site's framework Skill; for Next.js, use the SEO/GEO reference in Web frameworks.
• You need a general documentation audit or rewrite. Use ak:docs.
• Your source is only a live website or a single file. The bundled script accepts a local directory; inspect or stage approved sources first, or build a separate authorized workflow.
• You expect the file alone to guarantee indexing, retrieval, model support, or discoverability. Verify each consumer separately.

Prepare the source and publication contract

Before you start:

• Complete Onboarding, and confirm Engineer Kit is installed for the active runtime and scope.
• Identify the local documentation directory, public base URL, canonical route mapping, project name and description, output directory, and whether the full artifact is required.
• Check whether the site serves a Markdown variant of each page (.md appended to the page URL) and whether directory pages use a trailing slash.
• Exclude private, generated, duplicate, draft, translated fallback, internal, or machine-owned material that should not be published.
• Decide which topics are primary and which belong in the final ## Optional section for shorter context windows.
• Make Python 3 and permission to write the chosen output directory available.

See Runtime adapters for target-specific delivery behavior.

Run the Skill

Pass the source, output, public URL, curation constraints, and full-file choice in natural language.

/ak:llms "Scan ./content/docs/public for published Markdown only. Generate llms.txt and llms-full.txt in ./public using https://docs.example.com as the base URL, exclude drafts and generated release notes, and report broken or ambiguous route mappings without deploying."

/ak:llms "Scan ./content/docs/public for published Markdown only. Generate llms.txt and llms-full.txt in ./public using https://docs.example.com as the base URL, exclude drafts and generated release notes, and report broken or ambiguous route mappings without deploying."

$ak:llms "Scan ./content/docs/public for published Markdown only. Generate llms.txt and llms-full.txt in ./public using https://docs.example.com as the base URL, exclude drafts and generated release notes, and report broken or ambiguous route mappings without deploying."

The Skill arguments and the bundled Python script both use --base-url for the public URL prefix. For direct script execution, its interface is:

generate-llms-txt.py --source <directory> [--output <directory>] [--base-url <url>] [--md-links] [--trailing-slash] [--full] [--project-name <name>] [--project-description <description>]

--source must resolve to a directory. Without --base-url, generated links remain relative filesystem-style paths; that may be unsuitable for a file published at a website root. With a base URL, the script strips .md/.mdx and maps guide/index.md to guide, or to guide/ with --trailing-slash. --md-links links each page's Markdown variant instead of its HTML URL: the page URL plus .md, or index.html.md for a URL that ends in /. Use it only when the site actually serves those variants.

Understand the output contract

The tagged convention requires an H1. It recommends a short blockquote summary, grouped link lists, and a final ## Optional section for lower-priority material. These structural checks do not prove that a model or crawler will use the file.

For Markdown variants, the Skill asks that the HTML page keep rel=canonical and add <link rel="alternate" type="text/markdown"> pointing to its .md twin, that the variant contain only the page body with frontmatter, navigation, and components resolved to plain Markdown, and that variants, llms.txt, and llms-full.txt come from the same source at build or request time. If the site cannot serve .md URLs, the run links HTML URLs and reports the gap.

Observe the generation workflow

1. The run defines the public corpus. It confirms the local source, exclusions, canonical site routes, output location, and whether expanded content is safe to publish.
2. The run scans Markdown. The bundled script recursively reads .md and .mdx, skips hidden paths and node_modules, and ignores unreadable files.
3. The run extracts metadata. It uses the first H1 or a filename fallback for the title and the first meaningful paragraph for a truncated description.
4. The run categorizes pages. Path and filename heuristics place pages into getting started, guides, API reference, configuration, deployment, architecture, documentation, or optional groups.
5. The run generates artifacts. It writes llms.txt and, when approved, llms-full.txt to the chosen directory, linking .md variants when the site serves them.
6. The run curates and verifies. It checks headings, descriptions, category order, duplicates, URL mapping, public-content boundaries, artifact size, and links using the project's actual routes. With a live site, every linked URL should return 200 and sampled .md URLs should return text/markdown.

Keep source content and publication safe

• Generate into a reviewable workspace path before replacing a published file.
• Do not include absolute local paths, repository-only references, credentials, unpublished drafts, or private endpoints in either artifact.
• Treat source frontmatter and MDX as data to inspect. The full generator removes only a simple leading frontmatter form and may preserve imports, components, or syntax that is not useful as plain context.
• A large llms-full.txt consumes storage, bandwidth, and consumer context. Use the smallest public corpus that serves the intended reader.
• Ask separately before fetching live sites, writing outside the approved directory, committing, publishing, hosting, or deploying.

Verify the generated evidence

A complete run should provide:

• The source directory, exclusions, public base URL, route transformation, and project metadata used.
• Exact paths and sizes for llms.txt and any llms-full.txt output.
• Counts of scanned, included, excluded, unreadable, duplicate, and ambiguous pages where the available workflow can establish them.
• Evidence that the H1 exists, descriptions are useful, ## Optional is last, and each published link matches a real canonical route.
• A review of full-file content for secrets, private paths, imports, components, duplication, excessive size, and licensing concerns.
• Skipped link or consumer tests and the remaining publication decision.

The bundled script does not perform an HTTP link check or test multiple LLMs. Those are separate validation steps and must not be reported as complete unless they actually ran.

Troubleshoot or continue

Continue with Documentation for source improvements, or review Projects, artifacts, and checkpoints before replacing published files.

Know the current limits

• The bundled generator scans local directories only. It does not crawl sites, validate HTTP status, understand framework routing, serve or write .md page variants, deploy files, or measure model adoption.
• Metadata extraction and categorization are heuristic; MDX, localized trees, generated pages, index routes, and custom frontmatter need human review.
• The llms.txt convention itself can evolve, and consumer support varies. Check current requirements before relying on it as an integration contract.