GoProbe

A concurrent HTTP scanner built in Go. Throws a list of URLs at configurable worker pools, measures response latency, and tells you which one responds fastest.

Built as a systems exercise in Go concurrency — worker pools, context trees, goroutine lifecycle management, and memory-safe I/O.

Features

• Concurrent scanning via fixed worker pool (-c flag)
• Graceful shutdown on Ctrl+C via OS signal context propagation
• Per-request timeout with context chaining
• Memory-safe file ingestion with bufio.Scanner (handles million-line wordlists flat)
• TCP connection pooling via custom http.Transport
• Error logging to crawler_errors.log

Usage

# Single target
go run main.go -u https://example.com

# Target list with 20 concurrent workers
go run main.go -c 20 -f targets.txt

# Inline targets
go run main.go https://example.com https://google.com

Flags

Flag	Default	Description
-u	—	Single target URL
-f	—	Path to newline-separated target file
-c	5	Number of concurrent workers

Output

Prints the fastest responding URL and duration. Errors are silently logged to crawler_errors.log to keep stdout clean.