Skip to content

Commit 138e17a

Browse files
fix(docs): resolve llms generation review gaps (#1011)
1 parent 31bc0ce commit 138e17a

7 files changed

Lines changed: 239 additions & 362 deletions

File tree

‎.gitattributes‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ CHANGELOG.md linguist-documentation
2222

2323
# Lockfiles and large generated artefacts shouldn't pollute language stats.
2424
package-lock.json linguist-generated
25+
llms.txt linguist-generated
26+
llms-full.txt linguist-generated
2527
*.svg linguist-generated
2628
*.png binary
2729
*.jpg binary

‎.github/CONTRIBUTING.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,6 @@ We use AI assistants (Claude Code, GitHub Copilot, etc.) in our own development
4545

4646
We will not refuse a PR because it was AI-assisted, and we will not single it out either. See [AUTHORS.md](../docs/AUTHORS.md) for how we use AI ourselves.
4747
Read [AGENTS.md](../AGENTS.md) for AI coding agent commands and repository rules.
48-
After changing docs, run `node scripts/generate-llms-txt.mjs` and commit the generated files.
4948

5049
## Getting Started
5150

@@ -152,6 +151,8 @@ An adapter is one JSON file; the rest is generated and checked.
152151
4. Add `<region>/<slug>.live.spec.ts` (copy `intl/todoist.live.spec.ts`):
153152
static assertions always, live calls behind `RUN_<SLUG>_LIVE=1`.
154153
5. `node scripts/adapter-count.mjs --check` — bump the quoted counts it names.
154+
Then run `node scripts/generate-llms-txt.mjs` and commit the regenerated
155+
`llms.txt` and `llms-full.txt` alongside the count updates.
155156
6. Drop the brand logo at
156157
`packages/frontend/public/logos/connectors/<icon>.svg`.
157158

@@ -167,7 +168,9 @@ Open an issue before starting work on:
167168

168169
## Pull Request Process
169170

170-
1. **Update documentation** if your change affects user-facing behavior
171+
1. **Update documentation** if your change affects user-facing behavior.
172+
After changing docs, run `node scripts/generate-llms-txt.mjs` from the root
173+
and commit the generated files.
171174
2. **Add tests** for new functionality
172175
3. **Ensure all tests pass** (`npm test`)
173176
4. **Keep PRs focused** — one feature or fix per PR

‎AGENTS.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
# Agent instructions
22

33
Read [.github/CONTRIBUTING.md](.github/CONTRIBUTING.md) before making changes.
4-
After changing docs, run `node scripts/generate-llms-txt.mjs` and commit the generated files.
54
Never add or change code under any `ee/` directory: it uses the AnythingMCP
65
Commercial License; use AGPL-3.0-only for the rest of the repository.
76
Read [LICENSING.md](LICENSING.md), [the EE README](packages/backend/src/ee/README.md),
@@ -75,6 +74,10 @@ Record baseline failures separately from regressions; never claim a failed run p
7574
Avoid root `npm test`: it runs tests in all workspaces and fails because the
7675
frontend has no `test` script. Use the backend command above for catalog tests.
7776

77+
After changing docs, run `node scripts/generate-llms-txt.mjs` from the root and
78+
commit the generated files. Verify freshness with
79+
`node scripts/generate-llms-txt.mjs --check`.
80+
7881
Run frontend checks in `packages/frontend`:
7982

8083
```bash
@@ -127,7 +130,9 @@ Read [docs/tool-definition.md](docs/tool-definition.md); run adapter commands fr
127130
5. Check the quoted counts and update the files the checker names:
128131
```bash
129132
node scripts/adapter-count.mjs --check
133+
node scripts/generate-llms-txt.mjs
130134
```
135+
Commit the updated counts and regenerated `llms.txt` and `llms-full.txt`.
131136
6. Add the logo at `packages/frontend/public/logos/connectors/<icon>.svg`.
132137

133138
## Hard rules and deployment modes

‎llms-full.txt‎

Lines changed: 136 additions & 337 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎llms.txt‎

Lines changed: 4 additions & 5 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎scripts/generate-llms-txt.mjs‎

Lines changed: 17 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,24 @@
11
#!/usr/bin/env node
22
// Repository documentation for AI clients. No runtime or website output.
3-
import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
3+
import { existsSync, readdirSync, readFileSync, realpathSync, writeFileSync } from 'node:fs';
44
import { dirname, posix, resolve } from 'node:path';
55
import { fileURLToPath } from 'node:url';
66

77
const ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..');
88
const RAW = 'https://raw.githubusercontent.com/HelpCode-ai/anythingmcp/main/';
99
const GITHUB = 'https://github.com/HelpCode-ai/anythingmcp/blob/main/';
10+
// Contributor credits and Cloud operator internals are not product guides.
11+
export const EXCLUDED_DOCS = [
12+
'docs/AUTHORS.md',
13+
'docs/operations/card-trial-offer.md',
14+
'docs/operations/signup-attribution.md',
15+
];
1016
const SECTIONS = [
1117
['Getting started', ['README.md', 'docs/deployment.md', 'docs/guides.md']],
1218
['Connectors', ['docs/connectors/*.md', 'docs/tool-definition.md']],
1319
['AI clients', ['docs/integrations/*.md']],
1420
['Operations and security', ['docs/operations/*.md', 'docs/sso.md', 'docs/scim-entra-setup.md', 'docs/license-faq.md']],
15-
['Reference', ['docs/api-reference.md', 'docs/mcp-resources.md', 'docs/knowledge-graph.md']],
21+
['Reference', ['docs/api-reference.md', 'docs/mcp-resources.md', 'docs/knowledge-graph.md', 'docs/memory.md']],
1622
['Contributing', ['.github/CONTRIBUTING.md', 'AGENTS.md']],
1723
['Optional', ['docs/guides/*.md', 'docs/ROADMAP.md', 'docs/ADOPTERS.md']],
1824
];
@@ -21,6 +27,9 @@ const DESCRIPTIONS = {
2127
'docs/deployment.md': 'Deploy AnythingMCP with Docker and configure authentication, networking, and production settings.',
2228
'docs/connectors/login-token-auth.md': 'Build connectors that exchange credentials for cached bearer tokens with automatic refresh and re-login.',
2329
'docs/api-reference.md': 'REST API reference for authentication, connectors, tools, MCP servers, and administration.',
30+
'docs/tool-definition.md': 'Define MCP tool parameters, endpoint mappings, response transforms, and adapter metadata.',
31+
'docs/operations/observability.md': 'Configure structured logging, Sentry error reporting, and Prometheus metrics for your deployment.',
32+
'docs/ROADMAP.md': 'Review current development priorities, planned features, and contribution opportunities.',
2433
'docs/guides/connect-sorare-to-chatgpt.md': 'Connect Sorare fantasy football tools to ChatGPT through AnythingMCP.',
2534
'docs/guides/connect-sorare-to-claude.md': 'Connect Sorare fantasy football tools to Claude Desktop and Claude Code.',
2635
'docs/guides/connect-sorare-to-cloud.md': 'Use AnythingMCP Cloud to connect the Sorare GraphQL API without a local server.',
@@ -122,9 +131,9 @@ function renderHtml(text, inline = false) {
122131
}
123132

124133
function plain(text) {
125-
const entities = { '&amp;': '&', '&lt;': '<', '&gt;': '>' };
134+
const entities = { '&amp;': '&', '&lt;': '<', '&gt;': '>', '&nbsp;': ' ' };
126135
return renderHtml(text.replace(/\[([^\]]+)\]\([^)]*\)/g, '$1'), true)
127-
.replace(/[*`]/g, '').replace(/&(?:amp|lt|gt);/g, entity => entities[entity])
136+
.replace(/[*`]/g, '').replace(/&(?:amp|lt|gt|nbsp);/g, entity => entities[entity])
128137
.replace(/\s+/g, ' ').trim();
129138
}
130139

@@ -137,7 +146,7 @@ export function collectDocuments(root = ROOT) {
137146
return readdirSync(resolve(root, dir), { withFileTypes: true })
138147
.filter(entry => entry.isFile() && entry.name.endsWith('.md'))
139148
.map(entry => `${dir}/${entry.name}`).sort();
140-
}),
149+
}).filter(path => !EXCLUDED_DOCS.includes(path)),
141150
}));
142151
}
143152

@@ -214,8 +223,8 @@ export function cleanMarkdown(markdown, source) {
214223
.replace(/<p\b[^>]*>\s*<strong>(AnythingMCP[\s\S]*?)<\/strong>[\s\S]*?<\/p>/i,
215224
(_, summary) => `${plain(summary)}\n`);
216225
}
217-
text = renderHtml(text);
218-
return code.restore(rewriteProseLinks(text, source).replace(/\n{3,}/g, '\n\n').trim());
226+
text = renderHtml(text).replace(/&nbsp;/g, ' ');
227+
return code.restore(rewriteProseLinks(text, source).replace(/^[ \t]+$/gm, '').replace(/\n{3,}/g, '\n\n').trim());
219228
}
220229

221230
function metadata(content, path) {
@@ -247,7 +256,7 @@ export function generate(root = ROOT) {
247256
return { 'llms.txt': index, 'llms-full.txt': full };
248257
}
249258

250-
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
259+
if (process.argv[1] && realpathSync(process.argv[1]) === fileURLToPath(import.meta.url)) {
251260
try {
252261
if (process.argv.slice(2).some(arg => arg !== '--check')) throw new Error('Usage: node scripts/generate-llms-txt.mjs [--check]');
253262
const check = process.argv.includes('--check');

0 commit comments

Comments
 (0)