← News
/PLAYBOOK · 2026-09-13

Nginx Access Log Attack Forensics

Nginx access logs are first-hand evidence in attack tracing. Reusable flow: forensic log_format with timing and XFF, TOP IP and 404 stats, SQLi/path-traversal/scanner signatures, per-IP timeline, ipset and map bans, Fail2ban hookup, daily report script, and verify.

Nginx access logs are first-hand evidence in attack tracing. After a scan, CC, or WebShell upload, logs reconstruct attacker IP, timeline, paths, and tool fingerprints. Reusable flow below: forensic `log_format`, TOP IP and status stats, attack-signature ID, per-IP timeline, ban landing, daily report script, plus verify and FAQ.

Scope

Prerequisites

How it works

Nginx `access_log` writes one line at request end. Format is `log_format`. Default `combined`: client addr, time, request line, status, bytes, referer, User-Agent.

Forensic value is field cross-checks. `$status` splits normal vs probes (mass 404 is directory scan). `$request` shows payload. `$http_user_agent` shows scanner/script fingerprints. `$request_time` plus `$upstream_response_time` tells slow-client vs backend stall. Two caveats: `$http_user_agent` and `$http_x_forwarded_for` are client-spoofable — supporting evidence only; IP is the primary. Logs on the same host can be deleted or rewritten by a high-priv attacker, so forensics needs an off-box copy and append-only lock.

Procedure

Step 1: Define a forensic log format

In the `http` block of `/etc/nginx/nginx.conf`, define the format and attach it:

http {
    log_format forensic '$remote_addr - $remote_user [$time_local] "$request" '
                        '$status $body_bytes_sent "$http_referer" '
                        '"$http_user_agent" rt=$request_time '
                        'urt=$upstream_response_time xff="$http_x_forwarded_for"';

    access_log /var/log/nginx/forensic.log forensic buffer=64k flush=5s;
}

rt is total request time, urt is upstream time. Large gap → slow client. xff keeps the forward chain so you can reconcile the pre-CDN client. `buffer` + `flush` cut disk writes, at most 5s delay to disk. For live IR, drop them temporarily.

nginx -t && systemctl reload nginx

Step 2: TOP source IPs, status codes, paths

# TOP 20 IPs by volume
awk '{print $1}' /var/log/nginx/forensic.log | sort | uniq -c | sort -rn | head -20

# Status distribution
awk '{print $9}' /var/log/nginx/forensic.log | sort | uniq -c | sort -rn

# Paths with the most 404s (dir-scan signature)
awk '$9==404 {print $7}' /var/log/nginx/forensic.log | sort | uniq -c | sort -rn | head -20

# Per-IP request peak (by minute)
awk '/203.0.113.10/ {split($4,t,":"); print t[2]":"t[3]}' /var/log/nginx/forensic.log \
  | sort | uniq -c | sort -rn | head

Field positions match default combined: $1 client IP, $7 path, $9 status. If you moved `$request`, run `head -1 /var/log/nginx/forensic.log` and re-map fields.

Step 3: Identify attack signatures

Filter common payloads, aggregate to source IP:

grep -Ei "union.*select|sleep\(|benchmark\(|information_schema" /var/log/nginx/forensic.log \
  | awk '{print $1}' | sort | uniq -c | sort -rn | head

grep -E "\.\./|%2e%2e|/etc/passwd|/proc/self" /var/log/nginx/forensic.log \
  | awk '{print $1}' | sort | uniq -c | sort -rn | head

grep -Ei "wp-login|xmlrpc|\.php\?|shell|eval\(|base64_decode" /var/log/nginx/forensic.log \
  | awk '{print $1}' | sort | uniq -c | sort -rn | head

# Scanner and script UA fingerprints
awk -F'"' '{print $6}' /var/log/nginx/forensic.log | sort | uniq -c | sort -rn | head -30

Typical mapping: mass 404 plus empty UA or `python-requests`/`curl` → automated scan. Same IP, hundreds of dynamic pages in one minute, all 200, UA identical → CC or scraper. POST to a non-entry PHP, 200, abnormally large body → check immediately for a dropped WebShell.

Step 4: Full timeline for one IP

IP=203.0.113.10
grep -F "$IP" /var/log/nginx/forensic.log \
  | awk '{print $4,$5,$6,$7,$9,$11,$NF}' | head -50

Timeline answers three questions: first-seen (attack start), technique order (probe then exploit), whether it landed (200 + large body). If `error_log` is kept, align on request time to see backend exceptions:

grep -F "$IP" /var/log/nginx/error.log | tail -20

Step 5: Ban landing

Temporary ban: ipset + iptables. Constant-time, fine at hundreds of IPs:

ipset create wafban hash:ip timeout 86400 -exist
ipset add wafban 203.0.113.10 -exist
iptables -I INPUT -m set --match-set wafban src -j DROP
ipset list wafban | tail -20

Need Nginx 403 (keep access logs so you can watch): map.

# /etc/nginx/conf.d/blocked_ips.conf
# 203.0.113.10 1;

map $remote_addr $blocked_ip {
    default 0;
    include /etc/nginx/conf.d/blocked_ips.conf;
}

server {
    if ($blocked_ip) { return 403; }
}
nginx -t && systemctl reload nginx

Auto-ban on UA or request rate: Fail2ban. In `/etc/fail2ban/jail.d/nginx-scan.local` point `logpath` at forensic.log, custom filter on 404 and attack payloads. Stops hand-maintaining lists.

Step 6: Daily report script

#!/bin/bash
# /usr/local/bin/nginx_report.sh
LOG=/var/log/nginx/forensic.log
OUT=/var/log/nginx/report-$(date +%F).txt
{
  echo "=== $(date '+%F %T') TOP IP ==="
  awk '{print $1}' "$LOG" | sort | uniq -c | sort -rn | head -10
  echo "=== 404 TOP ==="
  awk '$9==404 {print $7}' "$LOG" | sort | uniq -c | sort -rn | head -10
  echo "=== Attack-signature hit IPs ==="
  grep -Ei "union.*select|\.\./|

Verify

FAQ

FAQ 1: Every IP in the log is a CDN node. How do I get the real client?

Enable `realip`, declare trusted sources, then read the right-hand XFF value:

set_real_ip_from 203.0.113.0/24;   # replace with CDN origin-pull ranges
real_ip_header X-Forwarded-For;
real_ip_recursive on;

After that `$remote_addr` is the real client. Do not put arbitrary public ranges in set_real_ip_from. Attacker-supplied XFF would forge source, and both logs and rate limits bypass.

FAQ 2: Attacker deleted access.log. Can I still trace?

Only if you have an off-box copy. Do three things: 1) remote syslog or a collector shipping `forensic.log` live to a separate log host; 2) append-only on the local file `chattr +a /var/log/nginx/forensic.log` so even high-priv processes cannot truncate; 3) logrotate via `rename`, not delete, so pre-delete copies survive. Already gone, no copy: try `debugfs` or `extundelete`. Success depends on subsequent writes.

FAQ 3: One IP is low volume but the site is slow. How do I find it?

Rank by time, not count. Slow requests by `rt=` desc: `awk -F'rt=' '{split($2,a," "); print a[1],$0}' forensic.log | sort -rn | head -20`. `urt` near `rt` → backend or DB. `urt` empty and `rt` large → classic slow client holding connections. Pair with `limit_conn` and timeouts.

FAQ 4: Log file growing too fast. How do I cap size?

logrotate, daily rotate + compress: `/etc/logrotate.d/nginx` with `daily`, `rotate 30`, `compress`, `delaycompress`, keep `kill -USR1 $(cat /run/nginx.pid)` in `postrotate` so Nginx switches the fd. Forensics retention can go 90+ days for compliance; move history off the system disk.

Summary

Nginx access-log forensics is complete fields + crossed dimensions + a conclusion you can act on. Custom `log_format` first (timing + forward chain). awk for IP, status, path. Payload signatures + UA fingerprints for technique. Close the loop with ipset, map, or Fail2ban. Every trace keeps a log copy and an operator record so the conclusion still holds on review.