Eval Marketplace

Comprehensive security evaluation tools for agent skills and MCP servers, powered by GitHub and Bright Data integrations.

Overview

This marketplace provides two specialized evaluation skills that utlize MCP servers:

• agent-skill-evaluator: Security and safety evaluation for agent skills (.skill files)
• mcp-evaluator: Security and privacy evaluation for MCP servers

Both skills automatically assess security vulnerabilities, privacy risks, community feedback, and provide actionable recommendations with detailed risk scoring.

Installation Options

Option 1: Install via Plugin Marketplace for Claude Code (Recommended)

1. Add Marketplace

/plugin marketplace add /path/to/eval-marketplace

Or from GitHub:

/plugin marketplace add github:jeredblu/eval-marketplace

2. Install Plugin

/plugin install evaluator-tools@eval-marketplace

Option 2: Download Individual Skills (Claude Code or Claude Desktop)

Download skills individually for manual installation:

Agent Skill Evaluator

1. Download: agent-skill-evaluator.zip
2. Extract the zip file
3. Move contents to ~/.claude/skills/agent-skill-evaluator/
4. Restart Claude Code

MCP Evaluator

1. Download: mcp-evaluator.zip
2. Extract the zip file
3. Move contents to ~/.claude/skills/mcp-evaluator/
4. Restart Claude Code

Note: These skills function best with recommended MCP servers, you'll need to manually configure the MCP servers (see Configuration section below).

Claude Desktop Installation

For Claude Desktop users:

1. Download: agent-skill-evaluator.zip or mcp-evaluator.zip
2. Open Claude Desktop
3. Go to Settings > Capabilities > Upload Skill
4. Select the downloaded zip file
5. Repeat for the second skill if desired

Configuration

The evaluator skills work best with two MCP servers. Both are optional but highly recommended for full functionality.

Recommended MCP Servers

GitHub MCP Server (Recommended)

• Enables direct GitHub repository access for analyzing skills and MCP servers
• Installation: @modelcontextprotocol/server-github
• Requires: GitHub Personal Access Token

Bright Data MCP Server (Recommended)

• Enables web scraping and Reddit access for community feedback analysis
• Installation: @brightdata/mcp
• Requires: Bright Data API token
• Note: Enable Pro Mode for Reddit scraping

Install and configure these MCP servers following their official installation instructions.

Usage

Agent Skill Evaluator

Evaluate the security of agent skills from various sources:

Evaluate this skill: https://github.com/username/skill-repo

Is this skill safe? https://example.com/my-skill.skill

Security assessment for this skill please: [attach .skill file]

The evaluator will:

• Download and extract the skill
• Analyze SKILL.md for prompt injections
• Review scripts for malicious code
• Search community feedback
• Generate comprehensive security report with risk scoring

MCP Server Evaluator

Evaluate the security of MCP servers:

Evaluate this MCP server: https://github.com/username/mcp-server

Is this MCP safe to use? https://github.com/org/mcp-repo

The evaluator will:

• Analyze repository metadata and activity
• Review code for security vulnerabilities
• Search for alternatives and comparisons
• Gather community feedback (including Reddit with Pro Mode)
• Generate detailed assessment with recommendations

Features

Agent Skill Evaluator

• Prompt injection detection
• Malicious code pattern matching
• Hidden instruction scanning
• Data exfiltration detection
• Community validation
• Risk scoring (0-100 scale)
• Actionable recommendations

MCP Server Evaluator

• Security vulnerability analysis
• Privacy risk assessment
• Code quality review
• Alternative server discovery
• Community feedback research (Reddit, forums, GitHub)
• Multi-dimensional scoring
• Usability assessment

Graceful Degradation

Skills work without MCP servers but with reduced functionality:

Scenario	Behavior
No GitHub MCP	Uses web scraping for repository access
No Bright Data	Uses built-in web search (limited)
No Pro Mode	No Reddit scraping, basic search only

Example: Without MCPs

User: "Evaluate this MCP: https://github.com/example/server"
Claude: Uses basic web scraping, can't access private repos, 
        limited Reddit data, slower analysis

Example: With MCPs

User: "Evaluate this MCP: https://github.com/example/server"
Claude: Direct repo access, full code review, Reddit community 
        feedback, comprehensive security scan

Requirements

• Claude Code or Claude Desktop
• GitHub Personal Access Token (recommended)
• Bright Data API token (recommended, for Reddit scraping)