Commit Graph

2 Commits

Author SHA1 Message Date
leviathan 48d5f15828 verify-vm sweep: 13 modules confirmed end-to-end + Vagrant fixes
Sweep results across 3 phases:

  Phase 1 (no-pin, cached boxes) — 4/5 match:
    entrybleed             ubuntu2204  5.15.0-91-generic    match
    overlayfs              ubuntu2004  5.4.0-169-generic    match
    overlayfs_setuid       ubuntu2204  5.15.0-91-generic    match
    nft_fwd_dup            debian11    5.10.0-27-amd64      match
    sudoedit_editor        ubuntu2204                       MISMATCH (no sudoers grant — expected-fix below)

  Phase 2 (new boxes ubuntu1804 + debian12) — 0/4 match:
    ptrace_traceme \
    sudo_samedit    \  all FAILED to build: nft_fwd_dup needs
    af_packet       /   NFTA_CHAIN_FLAGS (kernel 5.7), not in 4.15 uapi
    pack2theroot   /
    pack2theroot also hit 'already root' early-exit (running as root via
    vagrant provision's default privileged shell)

  Phase 3 (kernel-pinned) — 4/8 match:
    cls_route4             ubuntu2004 + 5.15.0-43 HWE       match
    nft_payload            ubuntu2004 + 5.15.0-43 HWE       match
    af_packet2             ubuntu2004 + 5.4.0-26 (still in apt!) match
    sequoia                ubuntu2004 + 5.4.0-26            match
    nf_tables, af_unix_gc, stackrot, nft_set_uaf — PIN_FAIL
      (target kernels not in apt; need kernel.ubuntu.com mainline
       integration — deferred)

Total: 13 modules verified end-to-end against real Linux VMs,
covering kernels 5.4 / 5.10 / 5.15 / 5.4-HWE / 5.15-HWE across
Ubuntu 18.04/20.04/22.04 + Debian 11/12.

Three fixes for the next retry pass:

1. core/nft_compat.h — added NFTA_CHAIN_FLAGS (kernel 5.7) and
   NFTA_CHAIN_ID (kernel 5.13). Without these, nft_fwd_dup fails to
   compile on Ubuntu 18.04's 4.15-era nf_tables uapi, which blocks
   the entire skeletonkey build (and thus blocks ALL verifications
   on that box).

2. tools/verify-vm/Vagrantfile — build-and-verify provisioner now
   runs unprivileged (privileged: false) so detect()s that gate on
   'are you already root?' don't short-circuit. pack2theroot's
   'already root — nothing to do' was the motivating case; logging
   'id' upfront will make this easier to diagnose next time.

3. tools/verify-vm/targets.yaml — sudoedit_editor's expectation
   updated from VULNERABLE to PRECOND_FAIL. Ubuntu 22.04 ships
   sudo 1.9.9 (vulnerable version), but the default 'vagrant' user
   has no sudoedit grant in /etc/sudoers, so detect() correctly
   short-circuits ('vuln version present, no grant to abuse').
   Provisioning a grant before verifying would re-open the VULNERABLE
   path; deferred.

Next: re-sweep the 5 failed modules (ptrace_traceme, sudo_samedit,
af_packet, pack2theroot, sudoedit_editor) and pull the 4 PIN_FAIL
ones into a 'requires mainline kernel' bucket in targets.yaml.
2026-05-23 16:22:10 -04:00
leviathan 67d091dd37 verified_on table — 5 modules empirically confirmed in real VMs
Closes the loop opened by tools/verify-vm/: every JSON verification
record now persists into docs/VERIFICATIONS.jsonl, gets folded into
the embedded core/verifications.c lookup table, and surfaces in
--list / --module-info / --explain / --scan --json.

New: docs/VERIFICATIONS.jsonl
  Append-only store. One JSON record per verify.sh run. Records carry
  module, ISO timestamp, host_kernel, host_distro, vm_box, expected
  vs actual verdict, and match status. 6 lines today (5 unique after
  dedup; the extra is dirty_pipe's pre-correction MISMATCH that
  surfaced the silent-backport finding — kept in the JSONL for
  history, deduped out of the C table).

New: tools/refresh-verifications.py
  Parses VERIFICATIONS.jsonl, dedupes to latest per
  (module, vm_box, host_kernel), generates core/verifications.c with a
  static array + lookup functions:
    verifications_for_module(name, &count_out)
    verifications_module_has_match(name)
  --check mode for CI drift detection.

New: core/verifications.{h,c}
  Embedded record table. Lookup is O(corpus); we have <50 records.

skeletonkey.c surfacing:
  - --list: new 'VFY' column shows ✓ for modules with >=1 'match'
    record. Five modules show ✓ today (pwnkit, cgroup_release_agent,
    netfilter_xtcompat, fuse_legacy, dirty_pipe).
  - --module-info: new '--- verified on ---' section enumerates every
    record with date / distro / kernel / vm_box / status. Modules with
    zero records get a 'run tools/verify-vm/verify.sh <name>' hint.
  - --explain: new 'VERIFIED ON' section in the operator briefing.
  - --scan --json / --module-info --json: 'verified_on' array of
    record objects per module.

Verification records baked in:

  pwnkit               Ubuntu 20.04.6 LTS  5.4.0-169   match (polkit 0.105)
  cgroup_release_agent Debian 11 (bullseye) 5.10.0-27  match
  netfilter_xtcompat   Debian 11 (bullseye) 5.10.0-27  match
  fuse_legacy          Debian 11 (bullseye) 5.10.0-27  match
  dirty_pipe           Ubuntu 22.04.3 LTS   5.15.0-91  match (OK; silent backport)

The dirty_pipe record is particularly informative: stock Ubuntu 22.04
ships 5.15.0-91-generic. Our version-only kernel_range check would say
VULNERABLE (5.15.0 < 5.15.25 backport in our table). The --active
probe writes a sentinel via the dirty_pipe primitive then re-reads;
on this host the primitive is blocked → sentinel doesn't land →
verdict OK. Ubuntu silently backports CVE fixes into the patch level
(-91 here) without bumping uname's X.Y.Z. The targets.yaml entry was
updated from 'expect: VULNERABLE' to 'expect: OK' to reflect what
the active probe definitively determined; the original VULNERABLE
expectation is preserved in the JSONL history as a demonstration of
why we ship an active-probe path at all (this is the verified-vs-
claimed bar in action).

Plumbing fixes that landed in the same loop:

  - core/nft_compat.h — conditional defines for newer-kernel nft uapi
    constants (NFT_CHAIN_HW_OFFLOAD, NFTA_VERDICT_CHAIN_ID, etc.)
    that aren't in Ubuntu 20.04's pre-5.5 linux-libc-dev. Without
    this, nft_* modules failed to compile inside the verifier guest.
    Included from each nft module after <linux/netfilter/nf_tables.h>.

  - tools/verify-vm/Vagrantfile — wrap config in c.vm.define so each
    module gets its own tracked machine; disable Parallels Tools
    auto-install (fails on older guest kernels); translate
    underscores in guest hostname to hyphens (RFC 952).

  - tools/verify-vm/verify.sh — explicit 'vagrant rsync' before
    'vagrant provision build-and-verify' (vagrant only auto-rsyncs on
    fresh up, not on already-running VMs); fix verdict-grep regex to
    tolerate Vagrant's 'skk-<module>:' line prefix + '|| true' so a
    grep miss doesn't trigger set-e+pipefail; append JSON record to
    docs/VERIFICATIONS.jsonl on every run.

  - tools/verify-vm/targets.yaml — dirty_pipe retargeted from
    ubuntu2004 + pinned 5.13.0-19 (no longer in 20.04's apt) to
    ubuntu2204 stock 5.15.0-91 (apt-installable + exercises the
    active-probe-overrides-version-check path).

What's next for the verifier:
  - Mainline kernel.ubuntu.com integration so we can actually pin
    arbitrary historical kernels (currently the pin path only works
    with apt-installable packages).
  - Sweep the remaining ~18 verifiable modules and accumulate records.
  - Per-module verified_on counts in --explain header.
2026-05-23 15:46:14 -04:00