Hello,
I have noticed that most instances keep the default robots.txt file, which allows any page to be indexed by search engines. As privacy is a concern for many users in the fediverse, I believe it would be nice to add a preference in users settings excluding you own user's page from search engines by adding it to the robots.txt.

• I searched or browsed the repo’s other issues to ensure this is not a duplicate.