robots.txt resource page: an empty Allow line does not disallow everything #569

The robots.txt resource page (https://digital.gov/resources/introduction-robots-txt-files) says:

Use Allow: to allow nothing. In other words, use it to disallow everything.

An Allow: line with no path doesn't block anything. Google's robots.txt documentation puts it this way:

When no path is specified, the rule is ignored.

Someone following the page to block a site would leave it open.

Suggested wording: "An Allow: line with no path has no effect. Use Disallow: / to block everything."

Sources:

• https://developers.google.com/search/docs/crawling-indexing/robots/robots_txt

I run Synapse.