tools/iamroot-fleet-scan.sh + docs/DETECTION_PLAYBOOK.md

iamroot-fleet-scan.sh — bash wrapper that scp's the iamroot binary to a host list, ssh-runs --scan --json on each, aggregates results into a single JSON document. Supports: - hosts list from file or stdin - user@host:port syntax - parallel xargs execution (default -P 4) - ssh key / extra ssh opts pass-through - --no-sudo for hosts where root isn't required - --summary-only to suppress per-host detail - --no-cleanup to leave the binary on disk Critical fix during smoke-test: iamroot's exit codes are SEMANTIC (0=OK, 2=VULNERABLE, 4=PRECOND_FAIL, 5=EXPLOIT_OK). The wrapper must NOT treat nonzero exit as a transport failure; success is defined by 'stdout contains valid JSON', failure by 'stdout empty'. Verified end-to-end on kctf-mgr → kctf-fuzz: fleet-scan reports ok=1, failed=0, summary.vulnerable groups by CVE: copy_fail_gcm, dirty_frag_esp×2, entrybleed. Per-host detail included. docs/DETECTION_PLAYBOOK.md — operational integration guide: - Lifecycle diagram (inventory → scan → fleet scan → deploy/mitigate/upgrade → monitor) - Recipes by team size: single host, small fleet, large fleet - SIEM integration patterns: Splunk, Elastic, Sigma - Auditd-event lookup commands per module key - VULNERABLE decision tree (patch vs mitigate vs compensate) - Mitigation revert procedures + side-effect table - False-positive tuning table per rule key - Pre-patch quarantine pattern - Maintenance contract / module-shipping SLA
2026-05-16 20:29:48 -04:00
parent 3eeee01f06
commit ea5d021f0c
3 changed files with 507 additions and 0 deletions
@@ -0,0 +1,302 @@
+# IAMROOT detection playbook
+
+Operational guide for blue teams using IAMROOT defensively. Pairs
+with `docs/DEFENDERS.md` (the "what" reference) — this is the "how to
+make it part of your daily ops" guide.
+
+## The lifecycle
+
+```
+              ┌─────────────┐
+              │  inventory  │  ← iamroot --list (what's bundled?)
+              └──────┬──────┘
+                     ▼
+              ┌─────────────┐
+              │    scan     │  ← iamroot --scan --json (what am I vulnerable to?)
+              └──────┬──────┘
+                     ▼
+              ┌─────────────┐
+              │  fleet scan │  ← iamroot-fleet-scan.sh hosts.txt
+              └──────┬──────┘
+                     ▼
+        ┌────────────┼────────────┐
+        ▼            ▼            ▼
+   ┌────────┐  ┌─────────┐  ┌──────────┐
+   │ deploy │  │ mitigate│  │  upgrade │  ← three responses
+   │  rules │  │ (pre-fix│  │ (kernel  │
+   │(SIEM)  │  │ stopgap)│  │  patch)  │
+   └────┬───┘  └─────┬───┘  └─────┬────┘
+        └────────────┼────────────┘
+                     ▼
+              ┌─────────────┐
+              │   monitor   │  ← ausearch -k iamroot-* / SIEM alerts
+              └─────────────┘
+```
+
+## Recipes by team size
+
+### Single host (workstation / single server)
+
+```bash
+# Daily/weekly hygiene check
+sudo iamroot --scan
+
+# If anything's VULNERABLE, deploy detections + apply mitigation
+sudo iamroot --detect-rules --format=auditd | sudo tee /etc/audit/rules.d/99-iamroot.rules
+sudo augenrules --load
+sudo iamroot --mitigate copy_fail   # or whichever module fired
+```
+
+### Small fleet (~10-100 hosts, SSH-reachable)
+
+Use `tools/iamroot-fleet-scan.sh`:
+
+```bash
+# Hosts list — one per line; user@host:port supported
+cat > hosts.txt <<EOF
+prod-web-01
+prod-web-02
+deploy@bastion-01
+ops@db-01:2222
+EOF
+
+# Scan; binary scp'd, run, cleaned up. Output is one JSON doc.
+./iamroot-fleet-scan.sh \
+    --binary ./iamroot \
+    --ssh-key ~/.ssh/ops_key \
+    --parallel 8 \
+    hosts.txt > fleet-scan-$(date +%F).json
+
+# Show me hosts with any VULNERABLE finding
+jq '.hosts[] | select(.scan.modules | map(.result == "VULNERABLE") | any) | .host' \
+   fleet-scan-*.json
+
+# Show summary across the fleet
+jq '.summary' fleet-scan-*.json
+```
+
+Output shape:
+
+```json
+{
+  "generated_at": "2026-05-16T22:00:00Z",
+  "n_hosts": 4,
+  "summary": {
+    "ok": 4,
+    "failed": 0,
+    "vulnerable": [
+      { "cve": "CVE-2024-1086", "name": "nf_tables", "count": 2 },
+      { "cve": "CVE-2023-0458", "name": "entrybleed", "count": 4 }
+    ]
+  },
+  "hosts": [...]
+}
+```
+
+### Larger fleet (>100 hosts)
+
+`iamroot-fleet-scan.sh` is intentionally simple (parallel ssh). For
+fleets too large for SSH-fan-out, wrap it in your config-management
+tool of choice:
+
+- **Ansible**: ship the binary via `copy:`, run via `command:`, parse
+  JSON with `jq` in a follow-on task
+- **SaltStack**: `cmd.run` returning JSON; `salt-call --return` to your
+  SIEM
+- **Fabric / Mitogen**: same shape, just Python-side
+
+Sample Ansible task:
+
+```yaml
+- name: scan with iamroot
+  copy:
+    src: iamroot
+    dest: /tmp/iamroot
+    mode: '0755'
+- name: run --scan --json
+  command: /tmp/iamroot --scan --json --no-color
+  register: scan
+  changed_when: false
+  failed_when: false        # iamroot exit codes are semantic, not errors
+- name: collect
+  set_fact:
+    iamroot_scan: "{{ scan.stdout | from_json }}"
+- name: cleanup
+  file:
+    path: /tmp/iamroot
+    state: absent
+```
+
+## SIEM integration patterns
+
+### Splunk
+
+```
+# splunk input config (inputs.conf)
+[script:///opt/iamroot/iamroot-cron-scan.sh]
+interval = 86400
+source = iamroot
+sourcetype = iamroot:scan
+```
+
+`iamroot-cron-scan.sh`:
+
+```bash
+#!/bin/bash
+/usr/local/bin/iamroot --scan --json --no-color
+```
+
+Search the indexed events:
+
+```spl
+index=iamroot sourcetype="iamroot:scan" modules{}.result=VULNERABLE
+| stats count by host modules{}.cve
+```
+
+### Elastic / OpenSearch
+
+Filebeat module reading the per-host scan JSON files (one per day),
+indexed into an `iamroot-*` index pattern. Standard Kibana
+visualization on `modules.cve` over time tracks vulnerability lifecycle.
+
+### Sigma → your platform
+
+```bash
+# Ship Sigma rules into your platform
+iamroot --detect-rules --format=sigma > /etc/sigma/iamroot.yml
+# Convert to your target (Sentinel, Elastic, etc.) via sigmac
+sigmac -t elastic /etc/sigma/iamroot.yml
+```
+
+## Day-to-day operational shape
+
+### What "good" looks like in the SIEM
+
+- Daily `iamroot --scan --json` from every host indexed
+- Trend dashboard: count of VULNERABLE results by CVE over time
+- Goal: every VULNERABLE → OK transition within SLA (e.g., 14 days for
+  patched-mainline bugs, 24h for actively-exploited)
+- Alert on: any host with a result not seen yesterday (could indicate
+  a config drift, a new install, or a disabled mitigation)
+
+### Auditd events from the embedded rules
+
+After deploying `iamroot --detect-rules --format=auditd`:
+
+```bash
+# By module key
+sudo ausearch -k iamroot-copy-fail -ts today
+sudo ausearch -k iamroot-dirty-pipe -ts today
+sudo ausearch -k iamroot-pwnkit -ts today
+sudo ausearch -k iamroot-nf-tables-userns -ts today
+sudo ausearch -k iamroot-overlayfs -ts today
+
+# Anything iamroot-tagged in the last hour
+sudo ausearch -k 'iamroot-*' -ts recent
+
+# Forward to syslog (rsyslog example)
+# /etc/rsyslog.d/iamroot.conf:
+:msg, contains, "iamroot-" @@your-siem.example.com:514
+```
+
+### When a VULNERABLE result fires
+
+Decision tree:
+
+```
+A scan reports VULNERABLE for module X
+│
+├── Q: Can I patch the underlying kernel / package?
+│   ├── YES → schedule patch window. In the meantime:
+│   │        iamroot --mitigate X (if supported)
+│   │        Verify auditd rule for X is loaded.
+│   │        Monitor for the rule key.
+│   └── NO (legacy LTS, embedded device, prod freeze) →
+│            iamroot --mitigate X (essential)
+│            Compensating control: tighten LSM (SELinux/AppArmor)
+│            Document in risk register
+│
+└── Q: Was this VULNERABLE before? When?
+    ├── First time → config drift; investigate why detection now
+    │                 produces this result
+    └── Persistent → mitigation isn't applied OR is being reverted
+                      by config management; fix the config baseline
+```
+
+### Mitigation reverts
+
+Mitigations can break legitimate functionality:
+
+| Mitigation | Side effect |
+|---|---|
+| `copy_fail` blacklist algif_aead | strongSwan / IPsec breaks |
+| `copy_fail` blacklist esp4/esp6 | IPsec breaks |
+| `copy_fail` blacklist rxrpc | AFS / kAFS clients break |
+| `copy_fail` AppArmor restrict userns=1 | bubblewrap, podman rootless break |
+
+If you applied a mitigation and now need to revert (e.g., the kernel
+patch has rolled out fleet-wide):
+
+```bash
+sudo iamroot --cleanup copy_fail
+# OR manually:
+sudo rm /etc/modprobe.d/dirtyfail-mitigations.conf
+sudo rm /etc/sysctl.d/99-dirtyfail-mitigations.conf
+# Reload affected modules / sysctls per your distro
+```
+
+## Common false positives + tuning
+
+| Rule key | False positive | Fix |
+|---|---|---|
+| `iamroot-copy-fail-afalg` | strongSwan, libcrypto using kernel crypto | `-F auid=` exclude service account UIDs |
+| `iamroot-dirty-pipe-splice` | nginx, HAProxy, kTLS | `-F gid!=33 -F gid!=99` exclude web service accounts |
+| `iamroot-pwnkit-execve` | gnome-software, polkit's own re-exec | Correlate by parent process; pkexec via gnome dbus is benign |
+| `iamroot-nf-tables-userns` | docker rootless, podman, snap confined apps | Whitelist known userns-using service GIDs |
+| `iamroot-overlayfs` | docker / containerd mounting overlayfs as root | The rule is intended for unprivileged-userns overlayfs mounts; add `-F auid>=1000` |
+
+## Pre-patch quarantine pattern
+
+If a CVE is in active exploitation and you can't patch immediately:
+
+```bash
+# Stage 1: detect
+sudo iamroot --scan --json | jq '.modules[] | select(.cve == "CVE-XXXX")'
+
+# Stage 2: mitigate (where supported)
+sudo iamroot --mitigate <module>
+
+# Stage 3: monitor — auditd rules already deployed
+sudo ausearch -k 'iamroot-*' -ts today | grep <module>
+
+# Stage 4: contain — temporarily restrict the trigger surface
+# e.g., for nf_tables CVE-2024-1086:
+echo 0 | sudo tee /proc/sys/kernel/unprivileged_userns_clone
+# OR
+sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=1
+
+# Stage 5: alert
+# When auditd or sigma rule fires, page on-call
+```
+
+## Maintenance contract
+
+When IAMROOT ships a new module:
+
+1. CI test passes on at least one vulnerable + patched kernel pair
+2. Detection rules ship alongside (auditd + sigma minimum)
+3. CVES.md row added with patch status
+4. NOTICE.md credits original researcher
+5. ROADMAP.md updated
+
+Treat these as the SLA for any blue-team-facing deliverable.
+
+## When you find a new false positive
+
+File an issue at https://github.com/KaraZajac/IAMROOT/issues with:
+- The exact ausearch line that fired
+- The legitimate process that produced it
+- Distro / kernel version
+
+Most false-positive fixes are a `-F` filter on the embedded rule —
+small, mergeable.
@@ -0,0 +1,205 @@
+#!/usr/bin/env bash
+# iamroot-fleet-scan — scan a host list with iamroot, aggregate results
+#
+# Usage:
+#   iamroot-fleet-scan.sh [OPTIONS] hosts.txt
+#   iamroot-fleet-scan.sh [OPTIONS] -                       # hosts on stdin
+#   iamroot-fleet-scan.sh [OPTIONS] -                       # one host per line
+#
+# Each line in the host list is either:
+#   - a hostname/IP (uses default ssh user from your config)
+#   - user@host
+#   - user@host:port
+#
+# Output: combined JSON to stdout, one object per host:
+#   { "generated_at": "...", "summary": {...},
+#     "hosts": [ { "host": "...", "ok": true,
+#                  "scan": { /* iamroot --scan --json */ } }, ... ] }
+#
+# Options:
+#   --binary <path>      path to iamroot binary (default: ./iamroot)
+#   --ssh-key <path>     ssh key file (passed to scp and ssh)
+#   --ssh-opts "..."     extra ssh options (e.g. "-o ConnectTimeout=5")
+#   --remote-path <p>    where to scp the binary (default: /tmp/iamroot)
+#   --no-sudo            don't prefix the remote command with sudo
+#   --parallel <N>       run N hosts concurrently (default: 4)
+#   --summary-only       skip per-host detail in stdout; print summary only
+#   --no-cleanup         leave the binary behind on each host (default: rm)
+#   -h | --help          this message
+#
+# Exit code: 0 if every host scanned (regardless of host-level vulns),
+#            1 if any host failed to scan.
+
+set -euo pipefail
+
+BINARY="./iamroot"
+SSH_KEY=""
+SSH_OPTS=""
+REMOTE_PATH="/tmp/iamroot"
+USE_SUDO=1
+PARALLEL=4
+SUMMARY_ONLY=0
+CLEANUP=1
+HOSTFILE=""
+
+usage() { sed -n '2,/^$/p' "$0" | sed 's/^# \?//'; exit "${1:-0}"; }
+
+while [[ $# -gt 0 ]]; do
+    case "$1" in
+        --binary)        BINARY="$2"; shift 2;;
+        --ssh-key)       SSH_KEY="$2"; shift 2;;
+        --ssh-opts)      SSH_OPTS="$2"; shift 2;;
+        --remote-path)   REMOTE_PATH="$2"; shift 2;;
+        --no-sudo)       USE_SUDO=0; shift;;
+        --parallel)      PARALLEL="$2"; shift 2;;
+        --summary-only)  SUMMARY_ONLY=1; shift;;
+        --no-cleanup)    CLEANUP=0; shift;;
+        -h|--help)       usage 0;;
+        -)               HOSTFILE="/dev/stdin"; shift;;
+        *)               HOSTFILE="$1"; shift;;
+    esac
+done
+
+if [[ -z "$HOSTFILE" ]]; then
+    echo "error: no host file provided. Use -h for help." >&2
+    exit 2
+fi
+
+if [[ ! -x "$BINARY" ]]; then
+    echo "error: iamroot binary not found / not executable: $BINARY" >&2
+    exit 2
+fi
+
+if ! command -v jq >/dev/null 2>&1; then
+    echo "error: jq is required for JSON aggregation" >&2
+    exit 2
+fi
+
+# Build ssh/scp option arrays
+SSH_BASE=(-o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=10)
+[[ -n "$SSH_KEY" ]]  && SSH_BASE+=(-i "$SSH_KEY")
+[[ -n "$SSH_OPTS" ]] && eval "SSH_BASE+=( $SSH_OPTS )"
+
+scan_one_host() {
+    local hostspec="$1"
+    local host port user
+    if [[ "$hostspec" == *:* ]]; then
+        port="${hostspec##*:}"
+        hostspec="${hostspec%:*}"
+    else
+        port="22"
+    fi
+    if [[ "$hostspec" == *@* ]]; then
+        user="${hostspec%@*}"
+        host="${hostspec#*@}"
+    else
+        user=""
+        host="$hostspec"
+    fi
+    local target="${user:+${user}@}${host}"
+
+    local sudo_prefix=""
+    [[ "$USE_SUDO" -eq 1 ]] && sudo_prefix="sudo"
+
+    # 1. scp the binary
+    if ! scp "${SSH_BASE[@]}" -P "$port" -q "$BINARY" \
+            "${target}:${REMOTE_PATH}" 2>/dev/null; then
+        echo "{\"host\":\"${hostspec}\",\"ok\":false,\"error\":\"scp failed\"}"
+        return 1
+    fi
+
+    # 2. run --scan --json
+    # iamroot's exit codes are SEMANTIC (0=OK, 2=VULNERABLE, 4=PRECOND_FAIL, etc.)
+    # — nonzero is NOT a failure here. Treat ANY stdout JSON as success;
+    # only ssh-transport-level failures (key denied, network) are real
+    # failures, and those manifest as empty stdout + nonzero exit.
+    local scan_out
+    scan_out=$(ssh "${SSH_BASE[@]}" -p "$port" "$target" \
+            "$sudo_prefix $REMOTE_PATH --scan --json --no-color" 2>/dev/null || true)
+    if [[ -z "$scan_out" ]]; then
+        echo "{\"host\":\"${hostspec}\",\"ok\":false,\"error\":\"ssh run failed (empty output)\"}"
+        # Still try to cleanup
+        [[ "$CLEANUP" -eq 1 ]] && ssh "${SSH_BASE[@]}" -p "$port" "$target" \
+            "rm -f $REMOTE_PATH" 2>/dev/null || true
+        return 1
+    fi
+
+    # 3. cleanup
+    if [[ "$CLEANUP" -eq 1 ]]; then
+        ssh "${SSH_BASE[@]}" -p "$port" "$target" \
+            "rm -f $REMOTE_PATH" 2>/dev/null || true
+    fi
+
+    # 4. emit one combined JSON object
+    if ! echo "$scan_out" | jq --arg h "$hostspec" \
+            '{host: $h, ok: true, scan: .}' 2>/dev/null; then
+        echo "{\"host\":\"${hostspec}\",\"ok\":false,\"error\":\"invalid JSON from iamroot\"}"
+        return 1
+    fi
+}
+
+# Read host list (strip comments, blank lines)
+mapfile -t HOSTS < <(grep -vE '^\s*(#|$)' "$HOSTFILE")
+if [[ ${#HOSTS[@]} -eq 0 ]]; then
+    echo "error: no hosts to scan" >&2
+    exit 2
+fi
+
+# Optional progress to stderr
+echo "[*] scanning ${#HOSTS[@]} host(s), parallel=$PARALLEL" >&2
+
+# Run in parallel with xargs. Each invocation prints one JSON object.
+export -f scan_one_host
+export BINARY SSH_BASE SSH_KEY REMOTE_PATH USE_SUDO CLEANUP
+# bash-export of an array doesn't survive, so re-serialize:
+export SSH_BASE_STR="${SSH_BASE[*]}"
+
+# Simpler: collect per-host results sequentially (good enough for small
+# fleets); parallel mode uses GNU xargs -P if available.
+TMP=$(mktemp)
+trap 'rm -f "$TMP"' EXIT
+
+if [[ "$PARALLEL" -gt 1 ]] && command -v xargs >/dev/null 2>&1; then
+    # -I{} implies -n1; specifying both warns on modern xargs.
+    printf '%s\n' "${HOSTS[@]}" | xargs -P"$PARALLEL" -I{} \
+        bash -c 'scan_one_host "$@"' _ {} >> "$TMP"
+else
+    for h in "${HOSTS[@]}"; do
+        scan_one_host "$h" >> "$TMP" || true
+    done
+fi
+
+# Aggregate. `jq -s` slurps the line-delimited JSON into an array.
+TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%SZ)
+RESULT=$(jq -s --arg ts "$TIMESTAMP" '
+    . as $hosts
+    | {
+        generated_at: $ts,
+        n_hosts: ($hosts | length),
+        summary: {
+            ok: ($hosts | map(select(.ok)) | length),
+            failed: ($hosts | map(select(.ok | not)) | length),
+            vulnerable: (
+                $hosts
+                | map(select(.ok))
+                | map(.scan.modules // [])
+                | flatten
+                | map(select(.result == "VULNERABLE"))
+                | group_by(.cve)
+                | map({cve: .[0].cve, name: .[0].name, count: length})
+                | sort_by(-.count)
+            )
+        },
+        hosts: $hosts
+    }
+' "$TMP")
+
+if [[ "$SUMMARY_ONLY" -eq 1 ]]; then
+    echo "$RESULT" | jq 'del(.hosts)'
+else
+    echo "$RESULT"
+fi
+
+# Exit nonzero if any host failed
+FAILED=$(echo "$RESULT" | jq -r '.summary.failed')
+[[ "$FAILED" -eq 0 ]] || exit 1