webgrip/infrastructure
5
0
Fork 0
Code Issues 1 Pull requests 2 Projects Releases 102 Packages Wiki Activity Actions

perf(cve-gate): run the gate as a step, not a container #90

Merged
ryangr0 merged 1 commit from perf/cve-gate-as-a-step into main 2026-08-07 11:38:19 +00:00
Conversation 0 Commits 1 Files changed 4 +167 -236
ryangr0 commented 2026-08-06 19:13:19 +00:00
Owner
Copy link

Cuts 262 lines to 107, of which the executable part is about 35. Merge after the ci-runner PR — see the order at the bottom.

The diagnosis

Five consecutive releases failed in this action, for five different reasons. Not one was about finding CVEs:

release failure
cve-gate 0.3.0 docker cp into a --read-only rootfs — refused by the daemon
run 221 gate-image pin naming a tag that was never built
run 224 seccomp clone rule that denied exactly what its comment said it allowed
run 234 2 GB tmpfs — too small for grype's DB plus an extracted multi-GB image
run 238 cache volumes created root:root, unwritable by uid 65532

They're the same failure wearing five different outfits: getting data into and out of a maximally hardened container reached through a remote dind daemon, where there's no shared filesystem, docker cp is refused, bind mounts resolve on the wrong host, and /tmp evaporates on exit.

That hardening was never required to run a scan. It was there because cve-gate doubles as the ADR-0006 worked example of a hardened image — and that second goal ended up on the critical path of every release.

What this deletes

ci-runner now bakes grype and cve-gate (both static Go, copied from that same image, digest-pinned), so the scan is a binary on PATH reading a workspace already on disk. Gone:

  • the docker pull per release
  • --read-only / --tmpfs / --cap-drop / --security-opt plumbing
  • the seccomp profile invocation and the verifier call
  • two cache volumes, and the root helper container that existed only to chown them
  • the ALLOW_MISSING_GATE bootstrap hatch
  • base64 marshalling of budgets and VEX into the environment
  • the stdout marker protocol and the awk that parsed the verdict back out of the log
  • the retry loop — which only existed because a registry hiccup inside all of that was fatal

Two things start working

Both were silently lost inside the container:

  • GITHUB_STEP_SUMMARY is in the environment, so the binary writes its own budget table. Inside the container that variable didn't exist.
  • OUT_DIR is a writable workspace, so cve-budget.json is written directly instead of being printed between markers and parsed back out.

What is deliberately kept

ops/security/seccomp/cve-gate.json and verify-profile.py stay. They're no longer applied by CI — and both the profile's first line and the techdocs page now say so. An unrun security control that looks applied is worse than none.

They remain the reference hardened invocation for running the cve-gate image standalone, which is what keeps it the ADR-0006 example. The image doesn't go away; it just leaves the hot path.

Verified

  • exit contract unchanged and still load-bearing: 0 met budget, 1 blew it, 2 no verdict. 2 is never a pass. All three paths confirmed to propagate.
  • the only caller (cosign-sign-attest) passes just image-ref and image-name — both retained, so no breakage.
  • verify-profile.py still passes 31/31 against the kept profile.

⚠️ Merge order

This requires the runner pool to be on a ci-runner that bakes the binaries:

  1. #91 (ci-runner bakes grype + cve-gate) → merge
  2. its release → ci-runner 1.2.3
  3. bump the ci-runner digest in homelab-cluster's forgejo-runner ScaledJob
  4. this PR

Landing this first would break every signing job. The PATH check fails with exactly that diagnosis — naming the ScaledJob digest as the fix — rather than cve-gate: command not found.

Cuts **262 lines to 107**, of which the executable part is about 35. Merge **after** the ci-runner PR — see the order at the bottom. ## The diagnosis Five consecutive releases failed in this action, for five different reasons. **Not one was about finding CVEs:** | release | failure | |---|---| | cve-gate 0.3.0 | `docker cp` into a `--read-only` rootfs — refused by the daemon | | run 221 | `gate-image` pin naming a tag that was never built | | run 224 | seccomp `clone` rule that denied exactly what its comment said it allowed | | run 234 | 2 GB tmpfs — too small for grype's DB plus an extracted multi-GB image | | run 238 | cache volumes created `root:root`, unwritable by uid 65532 | They're the same failure wearing five different outfits: **getting data into and out of a maximally hardened container reached through a *remote* dind daemon**, where there's no shared filesystem, `docker cp` is refused, bind mounts resolve on the wrong host, and `/tmp` evaporates on exit. That hardening was never required to run a scan. It was there because `cve-gate` doubles as the ADR-0006 *worked example* of a hardened image — and that second goal ended up on the critical path of every release. ## What this deletes `ci-runner` now bakes `grype` and `cve-gate` (both static Go, copied from that same image, digest-pinned), so the scan is a binary on PATH reading a workspace already on disk. Gone: - the `docker pull` per release - `--read-only` / `--tmpfs` / `--cap-drop` / `--security-opt` plumbing - the seccomp profile invocation **and** the verifier call - two cache volumes, and the root helper container that existed only to `chown` them - the `ALLOW_MISSING_GATE` bootstrap hatch - base64 marshalling of budgets and VEX into the environment - the stdout marker protocol and the `awk` that parsed the verdict back out of the log - the retry loop — which only existed because a registry hiccup inside all of that was fatal ## Two things start working Both were silently lost inside the container: - **`GITHUB_STEP_SUMMARY` is in the environment**, so the binary writes its own budget table. Inside the container that variable didn't exist. - **`OUT_DIR` is a writable workspace**, so `cve-budget.json` is written directly instead of being printed between markers and parsed back out. ## What is deliberately kept `ops/security/seccomp/cve-gate.json` and `verify-profile.py` stay. They're no longer applied by CI — **and both the profile's first line and the techdocs page now say so.** An unrun security control that looks applied is worse than none. They remain the reference hardened invocation for running the `cve-gate` image standalone, which is what keeps it the ADR-0006 example. The image doesn't go away; it just leaves the hot path. ## Verified - exit contract unchanged and still load-bearing: `0` met budget, `1` blew it, `2` no verdict. **2 is never a pass.** All three paths confirmed to propagate. - the only caller (`cosign-sign-attest`) passes just `image-ref` and `image-name` — both retained, so no breakage. - `verify-profile.py` still passes 31/31 against the kept profile. ## ⚠️ Merge order This **requires** the runner pool to be on a ci-runner that bakes the binaries: 1. **#91** (ci-runner bakes grype + cve-gate) → merge 2. its release → ci-runner 1.2.3 3. bump the ci-runner digest in homelab-cluster's `forgejo-runner` ScaledJob 4. **this PR** Landing this first would break every signing job. The PATH check fails with exactly that diagnosis — naming the ScaledJob digest as the fix — rather than `cve-gate: command not found`.
262 lines of action become 107, of which the executable part is about 35.

Five consecutive releases failed in this action, for five different reasons, and not one was about
finding CVEs:

  - `docker cp` into a --read-only rootfs, refused by the daemon (cve-gate 0.3.0 shipped unsigned)
  - a gate-image pin naming a tag that was never built (run 221)
  - a seccomp clone rule that denied exactly what its comment said it allowed (run 224)
  - a 2g tmpfs, too small for grype's DB plus an extracted multi-GB image (run 234)
  - cache volumes created root-owned, unwritable by uid 65532 (run 238)

All five are the same failure wearing different clothes: getting data into and out of a maximally
hardened container reached through a REMOTE dind daemon, where there is no shared filesystem,
`docker cp` is refused, bind mounts resolve on the wrong host, and /tmp evaporates on exit.

That hardening was never required to run a scan. It was there because cve-gate doubles as the
ADR-0006 worked example of a hardened image, and that second goal ended up on the critical path of
every release. ci-runner now bakes grype and cve-gate — both static Go, copied from that same
image, digest-pinned — so the scan is a binary on PATH reading a workspace already on disk.

Deleted: the docker pull, --read-only/--tmpfs/--cap-drop/--security-opt plumbing, the seccomp
profile invocation and its verifier call, two cache volumes, the root helper container that existed
only to chown them, the ALLOW_MISSING_GATE bootstrap hatch, the base64 marshalling of budgets and
VEX into the environment, the stdout marker protocol, the awk that parsed the verdict back out of
the log, and the retry loop that existed because a registry hiccup inside all of that was fatal.

Two things start working that could not before, both silently lost inside the container:
GITHUB_STEP_SUMMARY is in the environment, so the binary writes its own budget table; and OUT_DIR
is a writable workspace, so cve-budget.json is written directly.

The exit contract is unchanged and still the load-bearing part: 0 met its budget, 1 blew it, 2
could not reach a verdict. 2 is never a pass. Verified all three paths propagate correctly.

KEPT: ops/security/seccomp/cve-gate.json and verify-profile.py. They are no longer applied by CI,
and both the profile and the techdocs page now say so in the first line — an unrun security control
that looks applied is worse than none. They remain the reference hardened invocation for running
the cve-gate image standalone, which is what keeps it the ADR-0006 example.

REQUIRES the runner pool to be on a ci-runner that bakes the binaries. Merge order: the ci-runner
change, then its release, then bump the digest in homelab-cluster's forgejo-runner ScaledJob, then
this. Landing this first would break every signing job — so the PATH check fails with exactly that
diagnosis rather than "command not found".
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

  
released
No labels
pull-request
released
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
webgrip/infrastructure!90
No description provided.