Gobuster

Spider tool to scan website for hidden pages.

Table of Contents

Installation

Prerequisites: Install Go.

sudo apt install gobuster

Or:

cd /usr/bin
git clone https://github.com/OJ/gobuster.git
cd /gobuster
go run main.go

Help

gobuster dir --help
gobuster dns --help
gobuster vhost --help

Available commands

dir: Uses directory/file bruteforcing mode
dns: Uses DNS subdomain bruteforcing mode
help: Help about any command
vhost: Uses VHOST bruteforcing mode

dir mode

Search for directories listed in a wordlist on a website URL.

To whitelist specific status code (–status-codes), unset blacklist (–status-codes-blacklist ”). See this note.

Check if you find any sections restricted to certain roles. Scan using the GET method, then rescan using POST.

Behind a WAF:

  • Add a User-Agent: –useragent “Mozilla/5.0”
  • Use the default number of threads: –threads 2 –delay 0ms
  • If there is a whitelist on IP address, use a proxy:
    –proxy “http://proxy.my-company-domain.com:8080”

Directory scan for metadata

➡ Recommended scan

Use wordlist metadata.txt.

URL="https://example.com"
WL=/home/kali/Wordlists/web/metadata.txt
gobuster dir --url $URL --wordlist $WL --status-codes '200,204,301,302,307,403,500' --status-codes-blacklist '' --no-tls-validation --expanded --discover-backup --threads 2 --delay 0ms --useragent "Mozilla/5.0"

Directory scan without extensions

➡ Recommended scan

Use the small list (~80K words) from SecLists.

URL="https://example.com"
WL=/usr/share/dirbuster/wordlists/directory-list-2.3-small.txt
#WL=/usr/share/seclists/Discovery/Web-Content/directory-list-2.3-medium.txt
gobuster dir --url $URL --wordlist $WL --status-codes '200,204,301,302,307,403,500' --status-codes-blacklist '' --no-tls-validation --expanded --discover-backup --threads 2 --delay 0ms --useragent "Mozilla/5.0"
gobuster dir --url $URL --wordlist $WL --status-codes '200,204,301,302,307,403,500' --status-codes-blacklist '' --no-tls-validation --expanded --discover-backup --threads 2 --delay 0ms --useragent "Mozilla/5.0" --method POST

Directory scan with extensions

➡ Recommended scan

Add years 2015 to present in common.txt

URL="https://example.com"
WL=/usr/share/dirb/wordlists/common.txt
gobuster dir --url $URL --wordlist $WL --status-codes '200,204,301,302,307,403,500' --status-codes-blacklist '' --no-tls-validation --expanded --discover-backup --threads 2 --delay 0ms --useragent "Mozilla/5.0" -x txt,xml,json,php,jsp,asp,aspx,old,bak,conf,config,log,db,sql
gobuster dir --url $URL --wordlist $WL --status-codes '200,204,301,302,307,403,500' --status-codes-blacklist '' --no-tls-validation --expanded --discover-backup --threads 2 --delay 0ms --useragent "Mozilla/5.0" -x txt,xml,json,php,jsp,asp,aspx,old,bak,conf,config,log,db,sql --method POST

Other options

For specific web servers, use lists in /usr/share/seclists/Discovery/Web-Content/Web-Servers/

When using Microsoft IIS, MUST set asp,aspx because Gobuster will NOT find them otherwise.

-x php,txt,bak,old,aspx,asp

Exclude specific response length (size)

--exclude-length 11

Remove progress bar

--no-progress

Set number of threads and delay for each thread (default 10 threads, no delay)

--threads 4 --delay 100ms

Use a proxy

--proxy "http://127.0.0.1:3128"

Instead of adding proxychains in front of command, use the –proxy option.

--proxy socks5://127.0.0.1:8082

Using cookies

--cookies "cookie1=value1; cookie2=value2; cookie3=value3"

Using credentials over HTTP Basic Authentication & Proxy

-U username -P password

Using JWT token

TOKEN="jwt token here"
-H "Authorization: Bearer $TOKEN"

Search for directories listed in cgi.txt on website URL, use this wordlist:

WL=/usr/share/seclists/Discovery/Web-Content/LEGACY-SERVICES/CGIs/CGIs.txt

When HTTP 404 always returns 200, exclude requests with the same body length.

--status-codes '200,204,301,302,307,403,500' --status-codes-blacklist '' --exclude-length 23898,23868,23856,23850

fuzz mode

Add the keyword “FUZZ” in the URL and gobuster will replace it with the words from the wordlist.

wget https://raw.githubusercontent.com/carlospolop/Auto_Wordlists/main/wordlists/file_inclusion_windows.txt
wget https://gist.githubusercontent.com/korrosivesec/a339e376bae22fcfb7f858426094661e/raw/ec7d1167816a1b6a4a7843ddb72c94e1858d1b3a/lfi_windows.txt
wget https://raw.githubusercontent.com/carlospolop/Auto_Wordlists/main/wordlists/file_inclusion_linux.txt
URL="http://x.x.x.x/index.php?p=source&file=FUZZ"
WL=file_inclusion_windows.txt
gobuster --proxy socks5://127.0.0.1:8082 fuzz -u $URL -w $WL --timeout 30s -z -o gobuster-lfi.txt

s3 mode

See Amazon Simple Storage Service (Amazon S3).

gobuster s3 --help

Find S3 public buckets

If there is a whitelist on IP address, use a proxy:
–proxy “http://proxy.my-company-domain.com:8080”

gobuster s3 -w wordlist-of-bucket-names.txt

vhost mode

Uses VHOST enumeration mode (you most probably want to use the IP address as the URL parameter).

vhost scan – when infra is in scope

➡ Recommended scan

sudo wget -O /usr/share/wordlists/subdomains-top1million-20000.txt https://raw.githubusercontent.com/danielmiessler/SecLists/master/Discovery/DNS/subdomains-top1million-20000.txt
URL="https://example.com"
WL=/usr/share/wordlists/subdomains-top1million-20000.txt
gobuster vhost --url $URL --wordlist $WL --append-domain
gobuster vhost --url $URL --wordlist $WL --append-domain -m POST

dns mode

Uses DNS subdomain enumeration mode.

sudo wget -O /usr/share/wordlists/subdomains-top1million-20000.txt https://raw.githubusercontent.com/danielmiessler/SecLists/master/Discovery/DNS/subdomains-top1million-20000.txt
DOMAIN=example.com
WL=/usr/share/wordlists/subdomains-top1million-20000.txt
gobuster dns -q -r 8.8.8.8 -d $DOMAIN -w $WL