HTTP Crafter — Reconnaissance & Header Fingerprinting Detection
A single crafted request is indistinguishable from normal traffic and should not alert. Reconnaissance is a statistical signal, so the detection belongs in aggregation — request cadence, path diversity, and 404 ratio — not in per-request signatures.
Open in Red Team Toolkit — httpHTTP Crafter — Reconnaissance & Header Fingerprinting Detection
A single crafted request is indistinguishable from normal traffic and should not alert. Reconnaissance is a statistical signal, so the detection belongs in aggregation — request cadence, path diversity, and 404 ratio — not in per-request signatures.
Telemetry generated
| Log source | Field / identifier | What it shows |
|---|---|---|
| Web Access Log | method / path / user_agent / status / response_time | Primary reconnaissance signal. 404 volume and distribution is the clearest content-discovery indicator. Unusual HTTP methods (TRACE, PROPFIND, DEBUG) appear in the method field. User agent strings identify scanner tools but are trivially spoofed. |
| WAF / CDN Log | rate_limit_event / bot_score | Rate and bot-score events from edge infrastructure. A high bot score with low rate-limit triggers indicates a scanner operating just below the rate threshold — the behavioural profile query below catches this. |
| 404 and 405 Volume | status_code distribution | The clearest content-discovery signal. A source generating many 404s across diverse paths is enumerating the application surface. 405 Method Not Allowed spikes indicate method probing. |
Detection rules
# Rule 1: High distinct 404 URI stems per client IP over 5 minutes
title: Web Content Discovery — High 404 Rate per Source
id: d4e5f6a7-b8c9-0123-defa-234567890123
status: experimental
logsource:
category: webserver
detection:
selection:
sc-status: 404
timeframe: 5m
condition: selection | count(cs-uri-stem) by c-ip > 50
level: medium
tags:
- attack.reconnaissance
- attack.t1595.002
---
# Rule 2: Unusual HTTP methods indicating WebDAV or debug probing
title: Unusual HTTP Method — WebDAV or Debug Probe
id: e5f6a7b8-c9d0-1234-efab-345678901234
status: experimental
logsource:
category: webserver
detection:
selection:
cs-method|contains:
- 'TRACE'
- 'TRACK'
- 'PROPFIND'
- 'PROPPATCH'
- 'MKCOL'
- 'COPY'
- 'MOVE'
- 'SEARCH'
- 'DEBUG'
condition: selection
level: medium
tags:
- attack.reconnaissance
- attack.t1595.002Evasion & counters
Slowing the scan below rate thresholds so no single 5-minute window triggers the 404 rule.
Extend detection windows to hours or days as a scheduled hunt using cumulative distinct-404 counts. A source that generates 500 unique 404s over 48 hours is still a scanner even if each 5-minute window is quiet.
Rotating source IPs across a proxy pool so each source is below threshold.
Aggregate by REQUESTED PATH instead of source. A rarely-requested admin path (e.g. /wp-admin, /.env, /actuator) receiving hits from many different source IPs is anomalous even when each individual source is quiet.
Spoofing a legitimate browser user agent string to avoid the known-scanner-agent check.
User agent is a weak signal by design — weight the behavioural features (404 ratio, path diversity, request cadence) far more heavily. Correlate with the absence of asset requests a real browser would make: a "Chrome" user agent that never requests CSS, JS, or image files is not a browser.
Requesting only paths that exist (using prior knowledge of the application) to avoid generating 404s.
This requires prior knowledge and severely limits discovery scope, which is itself a defensive win. It also means the attacker's traffic blends with legitimate users — at which point the detection shifts to authentication anomalies and access pattern analysis.
Rule tuning
Preventive control
Suppress Server and X-Powered-By response headers to remove version disclosure. Use non-default session cookie names. Return a consistent 404 response for both nonexistent and forbidden paths — a 403 confirms the path exists. Apply edge rate limiting at the CDN or load balancer layer.