Description
GAU (GetAllURLs) fetches known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, Common Crawl, and URLScan for any given domain. It's an essential passive recon tool for discovering historical endpoints.
Installation
BASH
go install github.com/lc/gau/v2/cmd/gau@latest
Basic Usage
BASH
# Fetch all known URLs
gau target.com
# Save output
gau target.com --o urls.txt
# From stdin
echo "target.com" | gau
Advanced Usage
BASH
# Specific providers
gau --providers wayback,commoncrawl,otx target.com
# Blacklist extensions
gau target.com --blacklist png,jpg,gif,css,woff
# Filter by date
gau target.com --from 202301 --to 202312
# JSON output
gau target.com --json
# Multiple domains
cat domains.txt | gau --o all-urls.txt
Common Workflows
BASH
# Find parameterized URLs for testing
gau target.com | grep "=" | sort -u > params.txt
# Find interesting endpoints
gau target.com | grep -iE "(api|admin|login|dashboard|config)" | sort -u