suprnova-markdown

Markdown twins of your app's prose pages, for the clients that don't run JavaScript.

An Inertia app answers with <div id="app"></div> and a JSON payload. Google renders it; GPTBot, ClaudeBot, PerplexityBot and curl do not. This crate gives every prose URL a .md sibling serving the source it was rendered from, and makes that sibling discoverable instead of a convention callers have to know.

GET /docs/routing      →  text/html         the page
GET /docs/routing.md   →  text/markdown     the source it came from

Use

use suprnova::{FrameworkError, async_trait};
use suprnova_markdown::{MarkdownMiddleware, MarkdownSource};

struct Content;

#[async_trait]
impl MarkdownSource for Content {
    async fn markdown(&self, path: &str) -> Result<Option<String>, FrameworkError> {
        // The page's path, suffix already stripped: `/docs/routing`.
        let Some(slug) = path.strip_prefix("/docs/") else {
            return Ok(None);
        };
        Ok(std::fs::read_to_string(format!("content/docs/{slug}.md")).ok())
    }
}

global_middleware!(MarkdownMiddleware::new(Content));

That is the whole integration. No controller knows the twins exist. Ok(None), or an error with status 404, falls through to the normal routes. Any other error is logged and answered with a plain 500 that carries none of the error's text and is not cached, so a broken source never looks like a missing page. Only GET and HEAD consult the source; other methods go straight to the router.

Advertise the twin in the page's <head>:

suprnova_markdown::alternate_link("https://example.com/docs/routing")
// <link rel="alternate" type="text/markdown" href="https://example.com/docs/routing.md">

What your source must check

The middleware trusts your MarkdownSource completely. It runs as global middleware, before the router, so the route groups' sign-in and permission checks never run for a .md request, and by default it marks the response public, max-age=3600. So:

• Return Markdown only for pages anyone may read: published, public, not members-only. Check that on every request, inside markdown(), from the current data. A lookup that finds any article by its slug hands out drafts to whoever guesses the address.
• If a page can stop being public (unpublished, deleted, made private), keep shared caches from serving the old copy: .max_age(0), or wrap the response with Cache-Control: no-store.
• Return the lookup's errors rather than Ok(None): a failure then answers 500 instead of quietly showing whatever the router has for that URL.

The directory starter's src/seo/markdown.rs checks eligibility and caching this way (it predates 0.3.0, so it also wraps the method and error handling the plugin now does itself).

Where the convention comes from

The .md-appended URL is the llms.txt proposal's own suggestion (Jeremy Howard, 3 September 2024): provide "a clean markdown version of those pages at the same URL as the original page, but with .md appended" (llmstxt.org). Stripe, Cloudflare and Anthropic all publish an llms.txt; the twin convention is common enough to have picked up a name, the Markdown Twin Pattern, and static-site generators and Next.js / Vite plugins already automate it.

Nothing here is novel. What this crate adds is doing it for a server-rendered SPA - where the HTML genuinely carries no prose, so the twin is the only text a non-executing client can get - behind an interface narrow enough to be reused.

The defaults, and why

A distinct URL, not content negotiation. Serving different bodies at one URL based on User-Agent is cloaking and search engines treat it as such. Keying on Accept avoids that but is unreliable - few crawlers send Accept: text/markdown - and forces Vary: Accept on every HTML response. A separate URL is cacheable, linkable, and testable with curl.

rel="alternate" for discovery. The same relation an RSS feed uses to announce an alternative representation of the current document. A client that understands HTML link relations learns nothing new, and the twin stops being a URL convention callers must know in advance.

X-Robots-Tag: noindex on the twin, by default. It is the same content as the page; letting both compete splits ranking signals. .indexable() turns it off for the case where markdown is the canonical form. This one is a judgment call rather than part of the convention - plenty of implementations leave the twins indexable.

charset=utf-8, always. Technical prose is full of arrows and box-drawing. A client that guesses latin-1 renders them as mojibake.

Completing the picture

The twins are one part. An llms.txt that indexes them, and an llms-full.txt that inlines the corpus, are what turn a set of URLs into something an agent can consume in one fetch. Those are app-shaped rather than framework-shaped - they depend on how your content is organised - so this crate doesn't generate them, but the twins are what they should link to.

Status

Version 0.5.6 targets Suprnova 4.6.2 through tag = "v4.6.2" (commit 1c8b2d07b65b120337b06c975522f850a340a418). Consumers must name the framework the same way - the same Git URL and tag = "v4.6.2" - so middleware and request types share one crate identity; a rev for the same commit is a different identity. The plugin turns off the framework's default features, so the app's own feature list stays in charge. Suprnova 4.2.0 apps stay on 0.4.0; Suprnova 4.1.0 apps stay on 0.3.0; Suprnova 3.2.1 apps stay on 0.1.1. Requires Rust 1.94 or later. Extracted from suprnova.app, where the pattern runs in production - see /llms.txt, /llms-full.txt, and the .md twin of any manual chapter.

Upgrading

See UPGRADE.md for compatible framework pins, consumer checks and rollback.