Skip to content
This repository was archived by the owner on Nov 18, 2021. It is now read-only.
This repository was archived by the owner on Nov 18, 2021. It is now read-only.

GitHub prevents crawling of repository's Wiki pages - no Google search #1683

@kevinjwalters

Description

@kevinjwalters

GitHub currently has a robots.txt which is preventing crawling of the paths associated with the Wiki area for each and every repository. This is explicit and looks very intentional. I've asked about this (19-Oct-2019) and got no response, ticket number is 430217.

I've attached the current (27-Oct-2019) robots.txt file.

github.com.robots.20191027.txt

The gist of it:

Allow: /*/*/tree/master
Allow: /*/*/blob/master
...
Disallow: /*/*/wiki/*/*

I would like this to change to make the Wiki areas searchable using popular search engines.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions