GoSpider — Reconnaissance | Tools

Description

GoSpider is a fast web spider written in Go designed for crawling websites and extracting URLs, JavaScript files, forms, and other interesting endpoints from web pages.

Installation

BASH

go install github.com/jaeles-project/gospider@latest

Basic Usage

                        BASH
                        
                            # Basic crawl
gospider -s https://target.com

# Set crawl depth
gospider -s https://target.com -d 3

# Save output
gospider -s https://target.com -o output/

Advanced Usage

                        BASH
                        
                            # Include subdomains
gospider -s https://target.com --include-subs

# Concurrent crawling
gospider -s https://target.com -c 10 -t 10

# From file
gospider -S targets.txt -o output/ -c 10

# With cookies
gospider -s https://target.com --cookie "session=abc123"

# Blacklist specific extensions
gospider -s https://target.com --blacklist ".(png|jpg|gif)"

Common Workflows

                        BASH
                        
                            # Crawl and extract unique URLs
gospider -s https://target.com -d 3 --include-subs | grep -oP 'https?://[^\s]+' | sort -u

# Find all JS endpoints
gospider -s https://target.com -d 2 | grep "\.js" | sort -u