Skip to content

add robotsNoIndex parameter#900

Merged
adityatelange merged 3 commits intoadityatelange:masterfrom
martin2250:noindex_param
May 22, 2022
Merged

add robotsNoIndex parameter#900
adityatelange merged 3 commits intoadityatelange:masterfrom
martin2250:noindex_param

Conversation

@martin2250
Copy link
Copy Markdown
Contributor

@martin2250 martin2250 commented Apr 30, 2022

What does this PR change? What problem does it solve?
This PR adds the ability to use the noindex and nofollow parameters for specific pages in production mode, preventing e.g. the imprint from showing up in search engines. It also lists these pages in the robots.txt file.

Was the change discussed in an issue or in the Discussions before?
#712 also touches this line, but it adds other functionality as well and doesn't include robots.txt.

Cheers!

PR Checklist

  • This change adds/updates translations and I have used the template present here.
  • I have enabled maintainer edits for this PR.
  • I have verified that the code works as described/as intended.
  • This change adds a Social Icon which has a permissive license to use it.
  • This change does not include any CDN resources/links.
  • This change does not include any unrelated scripts such as bash and python scripts.
  • This change updates the overridden internal templates from HUGO's repository.

@Thunder33345
Copy link
Copy Markdown

Thunder33345 commented May 6, 2022 •
edited
Loading

You shouldnt use Disallow: in robots.txt as it will stop crawlers from seeing the noindex directive, thus leaving the possibility of site still being indexed anyways.

However, robots.txt disallow does not guarantee that a page will not appear in results: Google may still decide, based on external information such as incoming links, that it is relevant and show the URL in the results. If you wish to explicitly block a page from being indexed, use the noindex robots meta tag or X-Robots-Tag HTTP header. In this case, don't disallow the page in robots.txt, because the page must be crawled in order for the tag to be seen and obeyed.
Cite: Google's Robots FAQs

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented May 8, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@martin2250
Copy link
Copy Markdown
Contributor Author

martin2250 commented May 8, 2022

@Thunder33345 Thanks, good call!

@adityatelange adityatelange merged commit 7b9c836 into adityatelange:master May 22, 2022
romainx pushed a commit to romainx/hugo-PaperMod that referenced this pull request Jul 26, 2022
@martin2250 @romainx
Add robotsNoIndex parameter (adityatelange#900)
d1297cc 
- adds the ability to use the noindex and nofollow parameters for specific pages in production mode, preventing e.g. the imprint from showing up in search engines.
- Usage: In post front-matter add,
    ---
    robotsNoIndex: true
    ---
minyeamer pushed a commit to minyeamer/hugo-PaperMod that referenced this pull request Sep 16, 2022
@martin2250
Add robotsNoIndex parameter (adityatelange#900)
a569500 
- adds the ability to use the noindex and nofollow parameters for specific pages in production mode, preventing e.g. the imprint from showing up in search engines.
- Usage: In post front-matter add,
    ---
    robotsNoIndex: true
    ---
dunklecat pushed a commit to dunklecat/hugo-PaperModX that referenced this pull request Oct 8, 2022
@martin2250 @dunklecat
Add robotsNoIndex parameter (adityatelange#900)
9f327a5 
- adds the ability to use the noindex and nofollow parameters for specific pages in production mode, preventing e.g. the imprint from showing up in search engines.
- Usage: In post front-matter add,
    ---
    robotsNoIndex: true
    ---
kylethedeveloper pushed a commit to kylethedeveloper/hugo-PaperMod that referenced this pull request Feb 21, 2023
@martin2250
Add robotsNoIndex parameter (adityatelange#900)
908f1c7 
- adds the ability to use the noindex and nofollow parameters for specific pages in production mode, preventing e.g. the imprint from showing up in search engines.
- Usage: In post front-matter add,
    ---
    robotsNoIndex: true
    ---
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@adityatelange adityatelange adityatelange approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@martin2250 @Thunder33345 @adityatelange