Blacklisting bots

  • Resolved carbeck

    (@carbeck)


    7 years, 7 months ago

    Hi there,

    how do I blacklist certain bots, e.g. YandexMobileBot, since that comes with Javascript support and thus triggers counting page views? Activating “Skip tracking for referrers listed in the comment blacklist” and putting “YandexMobileBot” in my Comments Blacklist field in Settings > Discussion seems to have zero effect.

    Kind regards
    carbeck

    • This topic was modified 7 years, 7 months ago by carbeck.
Viewing 2 replies - 1 through 2 (of 2 total)
  • Plugin Support Stefan Kalscheuer

    (@stklcode)

    7 years, 7 months ago

    Hi carbeck,

    Reason is not the JS capability of these bots, but their user agent strings.

    Statify comes with a default filter that whitelists Windows, Linux, iPhone, etc.
    The mobile bots however match this to force mobile display instead of desktop version.

    Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B411 Safari/600.1.4 (compatible; YandexBot/3.0; +http://yandex.com/bots) — Indexing robot.

    This behavior should probably be changed in future versions to a more refined blacklist.

    Currently the only chance is a custom skip hook or a blacklist plugin to filter their IP subnet. (my blacklist extension does not feature user agent filter either at the moment)

    Regards,
    Stefan

    rpzine

    (@rpzine)

    7 years, 7 months ago

    Bot activity is ruining my Statify stats, too. Thanks to GDPR compliance I’dont know which bots are running amok right now ;). But Yandex promises that it sticks to the robots.txt: https://yandex.com/support/webmaster/controlling-robot/robots-txt.html You could give it a try!

Viewing 2 replies - 1 through 2 (of 2 total)

The topic ‘Blacklisting bots’ is closed to new replies.

Tags