AI Content Signals

Περιγραφή

AI Content Signals allows you to easily implement the Content Signals Policy in your WordPress site’s robots.txt file. This gives you more control over how AI crawlers and large language models (LLMs) can use your content.

What are Content Signals?

Content Signals is an extension to the robots.txt standard created by Cloudflare that lets you specify three types of permissions for AI crawlers:

  • search – Allow or deny search indexing and traditional search results
  • ai-input – Allow or deny using your content for real-time AI responses (RAG, AI Overviews)
  • ai-train – Allow or deny using your content for training AI models

Key Features

  • Easy-to-use settings page in WordPress admin
  • Set global defaults for all crawlers
  • Configure specific settings for individual AI bots (GPTBot, ClaudeBot, PerplexityBot, etc.)
  • Add custom bot User-Agents
  • Supports both physical and virtual robots.txt files
  • Option to create physical robots.txt with basic WordPress rules
  • Preview generated Content Signals before applying
  • Export and import settings as JSON for easy migration between sites
  • Optional legal text with EU Directive reference
  • Developer-friendly: filter hook to extend the predefined bots list
  • Works with existing robots.txt from SEO plugins
  • Automatic sitemap detection and inclusion

Supported Bots

The plugin includes predefined settings for 28 major AI crawlers:

  • OpenAI GPTBot, OAI-SearchBot, and ChatGPT-User
  • Anthropic ClaudeBot, Claude-Web, and anthropic-ai
  • Perplexity Bot and Perplexity-User
  • Google Extended (Gemini) and GoogleOther
  • Amazon Bot
  • Apple Extended
  • Meta/Facebook Bot and meta-externalagent
  • DuckDuckGo DuckAssistBot
  • Allen Institute AI2Bot
  • Mistral AI
  • ByteDance Bytespider
  • DeepSeek AI
  • xAI Grok
  • Huawei Pangu
  • Common Crawl, Cohere AI, Diffbot, You.com Bot, and more

Important Notice

Content Signals is a declarative standard – it expresses your preferences but does not technically enforce them. AI companies are not legally required to respect these signals, though the plugin includes legal text referencing EU copyright directives.

The IETF AI Preferences (AIPREF) Working Group is currently developing a formal standard based on similar concepts. This plugin implements the current Cloudflare Content Signals specification and will be updated as standards evolve.

This plugin works best when combined with other protection measures like traditional robots.txt rules and server-level bot management.

Support

Need help or have suggestions?

Love the plugin? Please leave us a 5-star review and help spread the word!

About AyudaWP.com

We are specialists in WordPress security, SEO, and performance optimization plugins. We create tools that solve real problems for WordPress site owners while maintaining the highest coding standards and accessibility requirements.

Στιγμιότυπα

  • Main settings page with global Content Signals configuration
  • Bot-specific settings for granular control
  • Custom bots section for adding new AI crawlers
  • Live preview of generated robots.txt content
  • Export, import, and restore default settings

Εγκατάσταση

  1. Upload the plugin files to /wp-content/plugins/ai-content-signals/, or install through the WordPress plugins screen
  2. Activate the plugin through the ‘Plugins’ screen in WordPress
  3. Go to Settings > AI Content Signals to configure your preferences
  4. Save settings to apply Content Signals to your robots.txt

Συχνές Ερωτήσεις

What happens if I have a physical robots.txt file?

The plugin will detect your physical robots.txt file and append Content Signals to the end of it. Make sure the file is writable (permissions 644 recommended).

Can the plugin create a robots.txt file if I don’t have one?

Yes! You can enable the option “Create physical robots.txt file if it doesn’t exist” in the settings. This will create a basic robots.txt with standard WordPress rules plus your Content Signals.

What rules are included in the automatically created robots.txt?

The plugin creates a basic robots.txt with these rules:
* Disallow /wp-admin/ (except admin-ajax.php)
* Disallow /wp-includes/
* Disallow /wp-content/plugins/
* Disallow /wp-content/themes/
* Disallow /wp-content/cache/
* Disallow readme.html and license.txt
* Automatically includes your WordPress sitemap if available

What if I use an SEO plugin that manages robots.txt?

No problem! The plugin uses WordPress’s robots_txt filter, so it will add Content Signals to the virtual robots.txt generated by your SEO plugin without conflicts.

Will AI crawlers actually respect these signals?

Content Signals is a proposed standard and not all AI companies have committed to respecting it. It’s a way to express your preferences, but not a guarantee of enforcement. However, major AI companies like OpenAI, Anthropic, and Google have shown interest in respecting such signals.

Can I use this with Cloudflare?

Yes! If you use Cloudflare’s managed robots.txt, you can still use this plugin. The settings will apply to your WordPress-generated robots.txt.

What are the default settings?

By default, all signals are set to “yes” (allowed) for all user-agents. The option to create a physical robots.txt is disabled by default. You can change any of these settings in the settings page.

How do I remove Content Signals completely?

Simply uninstall the plugin. The uninstall process will remove all settings and clean up your physical robots.txt if applicable.

Can I transfer settings between sites?

Yes! Use the Export button on the settings page to download a JSON file with your full configuration, then use Import on another site to restore it. This includes global settings, individual bot overrides, and custom bots.

Is Content Signals an official web standard?

Not yet. Cloudflare published the Content Signals Policy in September 2025 and released it under CC0. The IETF has since created the AI Preferences (AIPREF) Working Group to develop a formal standard based on similar concepts. The plugin implements the current Cloudflare specification and will be updated as standards evolve.

Can developers extend the bot list programmatically?

Yes. Use the ayudawp_acs_predefined_bots filter to add or remove bots without modifying the plugin. Add this to your theme’s functions.php or a custom plugin:

add_filter( 'ayudawp_acs_predefined_bots', function( $bots ) {
    // Add a new bot.
    $bots['MyCustomBot'] = 'My Custom AI Bot';

    // Remove an existing bot.
    unset( $bots['Omgilibot'] );

    return $bots;
} );

The array key is the User-Agent string used in robots.txt and the value is the label displayed in the settings page.

Do I need to manually edit my robots.txt file?

No! The plugin handles everything automatically. Just configure your preferences in the settings page and save.

Κριτικές

26 Νοεμβρίου 2025 1 απάντηση
I think it’s a good idea and it can help us define how we want AI to use the content of our web pages.
Ανάγνωση όλων των 3 κριτικών

Συνεισφέροντες & Προγραμματιστές

“AI Content Signals” είναι λογισμικό ανοιχτού κώδικα. Οι παρακάτω έχουν συνεισφέρει στη δημιουργία του.

Συντελεστές

Το “AI Content Signals” έχει μεταφραστεί σε 2 γλώσσες. Ευχαριστούμε τους μεταφραστές για τις συνεισφορές τους.

Μεταφράστε το “AI Content Signals” στην γλώσσα σας.

Ενδιαφέρεστε για την ανάπτυξη;

Περιηγηθείτε στον κώδικα, ανατρέξτε στο αποθετήριο SVN ή εγγραφείτε στο αρχείο καταγραφής αλλαγών ανάπτυξης μέσω RSS .

Σύνοψη αλλαγών

1.2.2

  • Fix: Fixed promotional banner button alignment on WordPress 7.0 due to core CSS specificity changes

1.2.1

Tested up to WordPress 7.0

1.2.0

  • New: Added export/import settings feature (JSON) for easy migration between sites
  • New: Added restore defaults button to reset all settings
  • New: Added 3 new AI bots: DeepSeekBot, GrokBot (xAI), and PanguBot (Huawei)
  • New: Added ayudawp_acs_predefined_bots filter hook for developers to extend the bots list
  • New: Added sticky table headers for the bots table on scroll
  • New: Added dashicon-flag to settings page title
  • Improved: Added new recommendation to promotional banner services
  • Fix: Fixed checkbox sanitization to work correctly with JSON imports
  • Improved: Refactored uninstall.php file to follow WordPress Coding Standards
  • Improved: Code organization with new export-import.php file

1.1.0

  • Added 8 new AI bots: OAI-SearchBot, Perplexity-User, Amazonbot, meta-externalagent, DuckAssistBot, AI2Bot, MistralAI-User, and Timpibot
  • Replaced static promotional box with dynamic banner showing random AyudaWP plugins and services
  • Improved code quality: replaced echo esc_html__() with esc_html_e() throughout
  • Removed inline styles in favor of CSS classes
  • Added capability check to admin notices
  • Unified null coalescing operator usage across all files
  • Removed unnecessary Domain Path header
  • General code cleanup and WordPress Coding Standards improvements

1.0.1

  • Tested up to WordPress 6.9

1.0.0

  • Initial release
  • Global Content Signals settings
  • Bot-specific configuration for 17 predefined AI crawlers
  • Custom bot support
  • Physical and virtual robots.txt handling
  • Option to create physical robots.txt with basic WordPress rules
  • Automatic sitemap detection and inclusion
  • Legal text option with EU Directive reference
  • Live preview of robots.txt output
  • Status notifications for file permissions