Gobuster
Gobuster finds what is not linked: hidden directories, forgotten backup files, admin panels and subdomains. It works by asking for thousands of paths quickly and telling you which ones exist.
--exclude-length or Gobuster is useless.What it is
Gobuster is a fast brute-forcer written in Go. It takes a URL and a wordlist and requests every entry, reporting which ones return an interesting status code.
It has four modes: dir for files and directories on one host, dns for subdomain enumeration, vhost for virtual hosts on the same IP, and fuzz for arbitrary placeholder positions. Go's concurrency is why it is fast — which is also why an unthrottled run can hammer a small server.
How to use Gobuster
Follow it exactly as you would in a real session — every command, every line of output, and what each result actually tells you. This is a terminal walkthrough.
Check for a wildcard response first
kali@errordex:~$ curl -s -o /dev/null -w "%{http_code}\n" http://10.10.10.5/thisdoesnotexist123
200 # A non-existent path returned 200 — the server lies. # Gobuster results will be meaningless unless we filter.
Run a small list first
kali@errordex:~$ gobuster dir -u http://10.10.10.5 -w /usr/share/seclists/Discovery/Web-Content/common.txt -t 10
/admin (Status: 301) [Size: 312] [--> /admin/] /backup (Status: 403) [Size: 279] /index.php (Status: 200) [Size: 1043] /.git (Status: 301) [Size: 309] [--> /.git/] /uploads (Status: 301) [Size: 316] # 301 = real directory 403 = exists but blocked 200 = page
Add extensions — backups and source files
kali@errordex:~$ gobuster dir -u http://10.10.10.5 -w list.txt -x php,bak,zip,txt -t 10
/config.php.bak (Status: 200) [Size: 421] /backup.zip (Status: 200) [Size: 2841] # A .bak next to a config file often contains database credentials
Read robots.txt and sitemap — free intelligence
kali@errordex:~$ curl -s http://10.10.10.5/robots.txt
User-agent: * Disallow: /admin/ Disallow: /internal/ Disallow: /backup/
Flags & options
| FLAG | WHAT IT DOES |
|---|---|
| dir | Directory and file enumeration mode. |
| dns | Subdomain enumeration mode. |
| vhost | Virtual host discovery on the same IP. |
| -u URL | Target URL. |
| -w list | Wordlist path. |
| -x php,html,bak | Extensions to append. |
| -t N | Threads. Default 10; 50+ will hammer a small server. |
| -b 404,403 | Status codes to hide. |
| --exclude-length N | Hide responses of this length — the wildcard fix. |
| -k | Skip TLS certificate verification. |
| -o file | Write results to a file. |
| -r | Follow redirects. |
Why it matters
The most valuable things on a web server are usually the ones nobody linked to: a staging folder, a .git directory, an old backup, an admin panel behind a guessable path. Gobuster finds those in minutes. It is also the fastest way to discover that a site returns the same page for every path — which tells you a lot about how it is built.
How it works
Gobuster builds a request for every line in the wordlist and sends them concurrently. The response status code tells it whether the path exists. Because it uses Go's goroutines, it can send thousands of requests while a human would manage dozens — which is why throttling matters.
The interesting results are not only 200s. A 301 usually means a real directory that redirects to a trailing slash. A 403 means the path exists but is blocked. A 401 means authentication is required. All three are leads.
Commands & syntax
# the everyday command gobuster dir -u https://target.local -w /usr/share/seclists/Discovery/Web-Content/common.txt # hide 404s, follow redirects, save output gobuster dir -u https://target.local -w list.txt -b 404 -o dirs.txt
# look for source and backup files too gobuster dir -u https://target.local -w list.txt -x php,html,txt,bak,zip # wildcard server? exclude by response length gobuster dir -u https://target.local -w list.txt --exclude-length 1043
gobuster dns -d target.local -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt -t 30
# show IPs found
gobuster dns -d target.local -w subs.txt --show-ips# several sites on one IP — try Host headers
gobuster vhost -u https://10.10.10.5 -w subs.txt --append-domain# -t is threads. Default 10; 50+ will hammer a small server
gobuster dir -u https://target.local -w list.txt -t 10 --delay 100mskali@errordex:~$ gobuster dir -u http://10.10.10.5 -w /usr/share/seclists/Discovery/Web-Content/common.txt -t 20 =============================================================== Gobuster v3.6 by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart) =============================================================== [+] Url: http://10.10.10.5 [+] Method: GET [+] Threads: 20 [+] Wordlist: /usr/share/seclists/.../common.txt [+] Negative Status codes: 404 =============================================================== Starting gobuster in directory enumeration mode =============================================================== /admin (Status: 301) [Size: 312] [--> http://10.10.10.5/admin/] /backup (Status: 403) [Size: 279] /index.php (Status: 200) [Size: 1043] /.git (Status: 301) [Size: 309] [--> http://10.10.10.5/.git/] /uploads (Status: 301) [Size: 316] /robots.txt (Status: 200) [Size: 84] =============================================================== # /backup is 403 — it exists but is blocked. Worth trying directly. # /.git is a 301 to a real directory. That is source code disclosure # if the objects are readable. # /robots.txt is 200 — always read it, it lists what they hid.
Common mistakes
Ignoring 403 responses
403 means the path exists and is protected. That is a finding, and often the start of an access-control test.
Not checking for wildcards
If every request returns 200, your results are meaningless. Confirm with a random path first.
Using one giant wordlist on a fragile server
20,000 concurrent requests can take a small site down. Start with -t 10.
Forgetting extensions
A .bak or .zip next to a config file is often more valuable than any page.
Skipping robots.txt and sitemap.xml
They hand you a list of paths the owner did not want indexed.
Not saving output
You will lose track of 60 hits across three scans.
Practice safely
Run Gobuster against your own lab: a DVWA or bWAPP container, or any VM you host. TryHackMe and Hack The Box boxes expect directory brute forcing as a standard early step.