Web Application Analysishttptoolkitprobesstatus-codetech-detectwafretryablehttpprojectdiscovery

httpx-toolkit

Fast and multi-purpose HTTP toolkit that runs multiple probers using retryablehttp library to maintain result reliability with increased threads. Supports probing hosts, URLs, and CIDRs with smart auto-fallback from HTTPS to HTTP.

Description

httpx-toolkit is a fast and multi-purpose HTTP toolkit developed by ProjectDiscovery, packaged in Kali Linux to avoid conflicts with python3-httpx. It enables running multiple HTTP-based probings with features like status code display, technology detection via Wappalyzer, content length, server headers, and response analysis. Designed for efficiency, it handles edge cases with retries and backoffs to manage WAFs while supporting high thread counts.

Use cases include reconnaissance for web applications, where it probes lists of hosts or URLs to gather HTTP response details, detect technologies, and filter results based on status codes, content lengths, or strings. It's ideal for large-scale scanning of hosts, CIDRs, or extracted domains from TLS/CSP probes, providing reliable results even under rate limiting or blocking conditions.

The tool's modular codebase facilitates contributions and custom configurations, with output options in plain text, CSV, or JSONL formats, including full response storage and redirect chains. It supports advanced features like VHost enumeration, HTTP/2 probing, and pipeline requests.

How It Works

httpx-toolkit uses the retryablehttp library to execute multiple concurrent HTTP probers with configurable threads (default 50) and rate limits (default 150 req/s). It performs smart protocol fallback from HTTPS to HTTP, handles retries, backoffs for WAF evasion, and processes inputs as hosts, URLs, or CIDRs. Probes extract response elements like status codes, headers, body stats, and perform specialized checks like favicon hashing, TLS data grabbing, CSP/TLS domain extraction, VHost enumeration, HTTP/2, and pipelining. Matchers and filters apply conditions on codes, lengths, strings, regex, line/word counts, and favicons to include or exclude results, with output serialization supporting JSONL with request/response chains.

Installation

bash
sudo apt install httpx-toolkit

Flags

-l, -list stringInput file containing list of hosts to process
-request stringFile containing raw request
-sc, -status-codeDisplay Status Code
-td, -tech-detectDisplay wappalyzer based technology detection
-cl, -content-lengthDisplay Content-Length
-server, -web-serverDisplay Server header
-ct, -content-typeDisplay Content-Type header
-lc, -line-countDisplay Response body line count
-wc, -word-countDisplay Response body word count
-rt, -response-timeDisplay the response time
-mc, -match-code stringMatch response with given status code (-mc 200,302)
-ml, -match-length stringMatch response with given content length (-ml 100,102)
-ms, -match-string stringMatch response with given string
-mr, -match-regex stringMatch response with specific regex
-er, -extract-regex stringDisplay response content with matched regex
-mlc, -match-line-count stringMatch Response body line count
-mwc, -match-word-count stringMatch Response body word count
-mfc, -match-favicon string[]Match response with specific favicon
-fc, -filter-code stringFilter response with given status code (-fc 403,401)
-fl, -filter-length stringFilter response with given content length (-fl 23,33)
-fs, -filter-string stringFilter response with specific string
-fe, -filter-regex stringFilter response with specific regex
-flc, -filter-line-count stringFilter Response body line count
-fwc, -filter-word-count stringFilter Response body word count
-ffc, -filter-favicon string[]Filter response with specific favicon
-t, -threads intNumber of threads (default 50)
-rl, -rate-limit intMaximum requests to send per second (default 150)
-faviconProbes for favicon ("favicon.ico" as path) and display phythonic hash
-tls-grabPerform TLS(SSL) data grabbing
-tls-probeSend HTTP probes on the extracted TLS domains
-csp-probeSend HTTP probes on the extracted CSP domains
-pipelineHTTP1.1 Pipeline probe
-http2HTTP2 probe
-vhostVHOST Probe
-p, -ports string[]Port to scan (nmap syntax: eg 1,2-10,11)
-o, -output stringfile to write output results
-sr, -store-responsestore http response to output directory
-srd, -store-response-dir stringstore http response to custom directory
-csvstore output in CSV format
-jsonstore output in JSONL(ines) format
-irr, -include-responseinclude http request/response in JSON output (-json only)
-include-chaininclude redirect http chain in JSON output (-json only)
-store-chaininclude http redirect chain in responses (-sr only)
-r, -resolvers string[]List of custom resolvers (file or comma separated)
-allow string[]Allowed list of IP/CIDR's to process (file or comma separated)
-deny string[]Denied list of IP/CIDR's to process (file or comma separated)
-random-agentEnable Random User-Agent to use (default true)
-H, -header string[]Custom Header to send with request
-http-proxy, -proxy stringHTTP Proxy, eg http://127.0.0.1:8080
-unsafeSend raw requests skipping golang normalization
-resumeResume scan using resume.cfg
-silentSilent mode
-v, -verboseVerbose mode
-versionDisplay version
-nc, -no-colorDisable color in output
-debugDebug mode
-debug-reqShow all sent requests
-debug-respShow all received responses
-statsDisplay scan statistic
-nf, -no-fallbackDisplay both probbed protocol (HTTPS and HTTP)
-nfs, -no-fallback-schemeProbe with input protocol scheme
-maxhr, -max-host-error intMax error count per host before skipping remaining path/s (default 30)
-ec, -exclude-cdnSkip full port scans for CDNs (only checks for 80,443)
-retries intNumber of retries
-timeout intTimeout in seconds (default 5)
-rsts, -response-size-to-save intMax response size to save in bytes (default 2147483647)
-rstr, -response-size-to-read intMax response size to read in bytes (default 2147483647)

Examples

Display help and all available flags
httpx-toolkit -h
Process list of hosts from input file
httpx-toolkit -l hosts.txt
Probe hosts for status code, technology detection, and content length
httpx-toolkit -l hosts.txt -sc -td -cl
Match responses with status codes 200 or 302
httpx-toolkit -l hosts.txt -mc 200,302
Filter out responses with status codes 403 or 401
httpx-toolkit -l hosts.txt -fc 403,401
Output results to file in JSONL format
httpx-toolkit -l hosts.txt -o results.txt -json
Use 100 threads and 200 req/s rate limit
httpx-toolkit -l hosts.txt -t 100 -rl 200
Perform favicon, TLS grab, and VHost probes
httpx-toolkit -l hosts.txt -favicon -tls-grab -vhost
Updated 2026-04-16kali.org ↗