Skip to content

Add 23 catalog adapters for AI, scraping, research data and CMS - #1353

Merged
helpcodeai merged 1 commit into
mainfrom
feat/catalog-gaps-block-27
Oct 30, 2026
Merged

helpcodeai merged 1 commit into
mainfrom
feat/catalog-gaps-block-27

Conversation

@helpcodeai

Copy link
Copy Markdown
Contributor

Summary

Adds 23 catalog adapters:

  • AI infrastructure: Groq, Cohere, Mem0, Langfuse, E2B
  • Web scraping: Bright Data, ScrapingBee, Browserless, Diffbot
  • Research and market data: Semantic Scholar, OpenAlex, SEC EDGAR, Alpha Vantage, Massive (formerly Polygon.io), CoinMarketCap
  • Security and IP lookup: VirusTotal, IPinfo
  • Product feedback, CMS and translation: Productboard, Canny, Storyblok, Prismic, Lokalise, Crowdin

Each adapter is mapped from the vendor's API reference, and from its OpenAPI file where one is published (docsUrl in each adapter). Official MCP servers, SDKs and community MCP servers on GitHub were read to confirm paths, auth, paging and field names; their code and descriptions were not copied. Deletes and other actions that cannot be undone install switched off.

Changes

New adapters (679 listed, up from 656; 34 keyless)

Adapter Auth Tools (off at install)
groq BEARER_TOKEN 9
cohere BEARER_TOKEN 17
mem0 API_KEY 16 (2)
langfuse BASIC_AUTH 20
e2b API_KEY 13 (1)
bright-data BEARER_TOKEN 19 (1)
scrapingbee BEARER_TOKEN 8
browserless QUERY_AUTH 10 (1)
diffbot QUERY_AUTH 11
semantic-scholar NONE 18
openalex NONE 16
sec-edgar NONE 8
alpha-vantage QUERY_AUTH 30
massive BEARER_TOKEN 33
coinmarketcap API_KEY 22
virustotal API_KEY 23 (2)
ipinfo NONE 21
productboard BEARER_TOKEN 23
canny NONE 29
storyblok API_KEY 30 (2)
prismic NONE 14
lokalise API_KEY 37 (1)
crowdin BEARER_TOKEN 44

Notes:

  • Paid calls: Groq, Cohere, Bright Data, ScrapingBee, Browserless and Diffbot calls say so in their descriptions and ask for confirmation. Following openrouter and openai, model calls that return an answer are marked read-only.
  • Keyless data APIs: Semantic Scholar, OpenAlex, IPinfo and Prismic use auth type NONE.
    • An optional key is sent only when it is set, and the header is left out when the variable is empty.
    • OpenAlex dropped its mailto polite pool in February 2026, so the instructions recommend its free key.
    • SEC EDGAR needs a User-Agent with a contact, which the user fills in as a required setting.
  • Massive: Polygon.io became Massive on 2025-10-30. The slug is massive, with "Polygon.io" as an alias, and the host is api.massive.com.
  • Langfuse: traces, sessions and daily metrics go through the v2 observations and metrics APIs. The v1 endpoints are removed from the current OpenAPI and leave Langfuse Cloud on 2026-11-16.
  • Productboard: API v2 only. v1 was sunset on 2026-07-08. Entity deletes are left out, because deleting a parent deletes its children.
  • E2B: sandbox lifecycle only. Running code inside a sandbox would send the team API key into a sandbox where generated code runs, and its process output is streamed, so it is left out.
  • VirusTotal: virustotal_scan_url installs switched off, because a submitted URL is shared with the VirusTotal community and cannot be withdrawn. There is no file upload.
  • Browserless: browserless_function, which runs your own Puppeteer code, installs switched off.
  • Canny: auth type NONE, with the API key in every JSON body, as gb/peoplehr does. An errorWhen rule turns Canny's {"error": ...} bodies into failed calls.
  • Alpha Vantage, SEC EDGAR, Diffbot: they report errors inside HTTP 200 answers, and errorWhen maps those to failures.
  • Official MCP servers: most of these vendors run one; the URLs are in the instructions where they help. These adapters use the REST APIs with a plain key or token.

Other changes

  • Catalog regenerated. Quoted counts updated in the READMEs, CITATION.cff, glama.json, package.json and server.json. llms.txt and llms-full.txt regenerated.
  • 23 logos added. Alpha Vantage and SEC EDGAR publish no clean SVG mark, so they get simple text badges like fred.svg.

Cloud and self-hosted: the adapters behave the same in both modes; none is selfHostOnly.

Type

  • Bug fix
  • New feature
  • Documentation
  • Refactoring
  • Performance improvement
  • Other (describe below)

Testing

  • Added/updated unit tests
  • Tested manually (describe steps below)
  • No tests needed (explain why)

Each new adapter has a *.live.spec.ts with static assertions:

  • auth and base URL
  • the key endpoint mappings
  • tool prefixes, and that every declared parameter reaches the request
  • required path parameters, the probe, the instructions coverage and the exact switched-off set

Live parts are gated behind RUN_<SLUG>_LIVE=1 and only read.

Live reads run for real:

  • Keyless: Semantic Scholar, OpenAlex, SEC EDGAR (with a test User-Agent), IPinfo (own IP, 8.8.8.8) and Prismic (the public slicemachine-blank repository).
  • Alpha Vantage: the demo key that Alpha Vantage publishes.
  • Others: routes were checked with GET requests and an invalid key: 401 on mapped routes and 404 on unknown ones.
    • Canny's API is POST only, so its routes rest on the docs.
    • Browserless, ScrapingBee and Diffbot answer 401 to any path, so their routes rest on the docs and the official clients.

Commands, run from the repository root unless noted:

  • node scripts/validate-adapters.mjs --warn: exit 0, no errors or warnings for the new adapters
  • node scripts/regenerate-catalog.mjs: 693 adapters across 15 regions
  • node scripts/adapter-count.mjs --check: OK, 679 adapters, 34 keyless, 12239 tools
  • node scripts/generate-llms-txt.mjs --check: exit 0
  • In packages/backend:
    • npx jest src/adapters: 672 suites passed, 1 skipped; 24,033 tests passed, 1,770 skipped
    • npx tsc --noEmit -p tsconfig.json: exit 0
    • npx eslint <the 23 new specs>: exit 0

Checklist

  • My code follows the existing code style
  • I have tested my changes locally
  • I have updated documentation (if applicable)
  • All existing tests pass (npm test): adapter suites run locally (above); the full backend suite runs in CI
  • This PR has a descriptive title

Related Issues

Website guides follow in a separate website PR.

@helpcodeai
helpcodeai merged commit b346639 into main Oct 30, 2026
15 checks passed
@helpcodeai
helpcodeai deleted the feat/catalog-gaps-block-27 branch October 30, 2026 23:44
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 30, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant