WayRobots

A tool to find snapshots of robots.txt file stored at http://web.archive.org

Usage

Before you use make sure to have the required python libraries

$ pip install requests
$ pip install colorama

Basic use

$ python wayrobots.py -i target.com -y 2015-2019

Help page

$ python wayrobots.py -h