Unblock good AI bots
-
Hi,
Your plugin is awesome just 1 thing – bots like ahrefs, semrush are whitelisted but good bots like gpt, perplexity, claude and many more are not whitelisted. Can you also whitelist them?
The page I need help with: [log in to see the link]
-
Hi,
Thank you for your kind words and for the great suggestion – we really appreciate this type of feedback.
You’re absolutely right: today many AI-related crawlers and assistants (such as GPT, Perplexity, Claude, and others) play an important role alongside traditional SEO bots like Ahrefs or Semrush.
We’re already working on expanding our rules and settings to properly handle AI and neural network bots. In one of the upcoming updates, we plan to introduce dedicated options that will allow our users to explicitly choose whether to allow or block these AI bots, depending on their own goals and policies.
This approach will give site owners full control – from maximum openness for AI indexing and analysis, to strict blocking if needed.
Thanks again for taking the time to reach out and help us improve the plugin. Stay tuned – this functionality is coming soon.
Best regards,
BotBlocker TeamThat is a good news that you are adding more rules. Adding to my last comment i was troubled with random 404s generation by bad bots since past 2 months – no plugin paid or free was able to fix my issue. I stumbled on you by deperate need to fix this problem and guess what. I am a happy user now.
Also, all the recaptcha that the botblocker plugin provides have issues – when the captcha appears to users even when it is solved nothing happens – you still remain on the captcha solving page. You have to manually reload the page to access the page. Current solution i got it using google captcha v2.-
This reply was modified 1 month, 2 weeks ago by
Rajat Singh.
Hello,
Thank you for the kind words – we’re really glad to hear that BotBlocker helped you finally stop the attacks caused by bad bots. That kind of feedback honestly means a lot to us.
Regarding the captcha behavior you mentioned, please note an important detail.
Our captcha has a built-in timeout mechanism. If a user does not complete any action within 10 seconds, the captcha is automatically considered expired and will be triggered again. In this case, even if the captcha appears to be solved, the page will not proceed until the request is refreshed.
To resolve this, please try adjusting the Captcha Timeout (seconds) setting.
You can find it in the BotBlocker Captcha tab. Increasing this value usually fixes the issue for real users with slower interaction or network latency.If you still notice any unusual behavior after changing this setting, feel free to let us know – we’ll be happy to take a closer look.
Best regards,
BotBlocker TeamHello,
Just a quick update – we’ve released a new update that improves handling of AI and LLM-based crawlers.
The plugin now includes extended detection and whitelist support for major AI bots such as GPTBot (OpenAI), ClaudeBot (Anthropic), Gemini (Google), and several other large language model crawlers. This allows you to explicitly allow them without relying on generic rules.
You can review and manage these bots directly in the updated bot rules section.
Thank you again for the suggestion – your feedback helped us prioritize this improvement.
Best regards,
BotBlocker Team -
This reply was modified 1 month, 2 weeks ago by
You must be logged in to reply to this topic.