MCP Server

Connect AI agents to Spider through the Model Context Protocol. Crawl, scrape, search, and extract web data from Claude, Cursor, Windsurf, and any MCP-compatible client.

Install

The MCP server runs via npx with no installation required. Set your API key as SPIDER_API_KEY in your environment.

Add Spider MCP server to Claude Code

claude mcp add spider -- npx -y spider-cloud-mcp

Available Tools

The MCP server exposes 13 tools across two categories.

Core Tools

spider_crawl: Crawl a website and extract content from multiple pages
spider_scrape: Scrape a single page and extract its content
spider_search: Search the web and optionally crawl results
spider_links: Extract all links from a page
spider_screenshot: Capture page screenshots
spider_unblocker: Access blocked content with anti-bot bypass
spider_transform: Transform HTML to markdown or text
spider_get_credits: Check your API credit balance

AI Tools (Subscription Required)

spider_ai_crawl: AI-guided crawling using natural language prompts
spider_ai_scrape: AI-powered structured data extraction
spider_ai_search: AI-enhanced semantic web search
spider_ai_browser: AI-powered browser automation
spider_ai_links: AI-powered intelligent link extraction

Usage Example

Once configured, ask your AI agent to use Spider tools directly in conversation.

Example conversation with an MCP-connected AI agent

User: "Scrape spider.cloud and give me the pricing details"

Agent uses spider_scrape with:
  url: "https://spider.cloud"
  return_format: "markdown"

User: "Crawl the docs and find all API endpoints"

Agent uses spider_crawl with:
  url: "https://spider.cloud/docs"
  limit: 50
  return_format: "markdown"

Parameters

Each tool accepts the same parameters as the corresponding Spider API endpoint. Common parameters include url, return_format, limit, depth, proxy_enabled, and request. See the API reference for the full list.

npm Package

The server is published as spider-cloud-mcp on npm. Source code is available on GitHub.