feat(cve-gate): hardened gate image, CVE budgets and OpenVEX #74

Merged
ryangr0 merged 1 commit from fix/cve-gate-dhi-registry into main 2026-07-31 16:53:14 +00:00
Owner

Ships the CVE budget gate and its dependencies. Cut from main, so it also carries the digest-pin commit that PR #71 missed.

Releases exactly one image: cve-gate. No existing image directory is touched — verified: git diff --name-only main...HEAD | grep '^ops/docker/' returns only ops/docker/cve-gate. The OCI-label repoint that touches all 17 Dockerfiles is deliberately held back for a separate, batched PR, because on_source_change builds one matrix entry per changed dir and 17 concurrent builds would land on fringe-workstation.

What's in it

  • ops/docker/cve-gate — the gate, and the worked example for ADR-0006. DHI alpine-base with the -dev variant as a build stage only; grype copied from dhi.io/grype instead of curl | sh of an install script from a mutable branch; non-root 65532; read-only rootfs; build-time assertions that run as the nonroot user.
  • ops/security/cve-budgets.yaml — per-image ceilings that ratchet. New images start in warn, the pipeline measures them, the budget is set at the observed number, then it only goes down. ci-runner (8C/136H) and agent-runner (1C/51H) seeded from measured Trivy Operator data. cve-gate is the only entry starting at enforce 0/0.
  • ops/vex/ — OpenVEX as the only sanctioned suppression mechanism: hand-authored, PR-reviewed, justified from the closed vocabulary. Harbor's project-wide cve_allowlist stays empty.
  • .forgejo/actions/cve-gate — complete but not yet called. Wiring the call site in the same change that first builds the image would make cve-gate's own release depend on an image that doesn't exist. Step 2 restores it.
  • syft pinned to cyclonedx-json@1.6 — unpinned it emits 1.7; Dependency-Track tops out at 1.6 and returned 400 on every upload since run 133 (~50 releases), fail-soft, so nobody saw it.
  • Dependency-Track failures split by class — 5xx/000 (outage) stays a warning; 4xx (we sent something bad) becomes ::error::. Loud, still non-fatal.
  • ops/kyverno/ deleted — it targeted ghcr.io/webgrip/* with a Fulcio identity: a registry we no longer publish to, verified by a mechanism we no longer use. The live policy in homelab-cluster is correct; this copy was fiction that read as a guarantee.
  • ADR-0004 supersedes ADR-0002 (keyless/GitHub → OpenBao Transit), stating the regressions plainly: no SLSA provenance, no transparency log. ADR-0005 (VEX + budgets), ADR-0006 (hardened bases, proposed).

Build history — four real failures, all caught by assertions

Run Failure Fix
188 401 from dhi.io REGISTRY_DHI defaults to the Harbor proxy
192 jq relocation errors copy jq's shared libraries, not just the binary
194 same errors, /out/lib empty musl's ldd writes to stderr; 2>/dev/null discarded it

Run 188's real lesson: this repo's pattern is default to the public registry, override to Harbor via --build-arg. That is only safe because docker.io / ghcr.io / mcr.microsoft.com serve anonymously — a missing override just loses the cache. dhi.io refuses anonymous pulls, so the same pattern turns a missing flag into a hard failure. And it would keep being missing: dockerVerifyGate() in @webgrip/semantic-release-config hardcodes its own three --build-arg flags in another repo, and run 188 died in exactly that path. Inverting the default fixes both build paths at once.

Run 194's real lesson: an empty result was accepted as an answer. "ldd returned nothing" and "this binary needs nothing" behaved identically, so the stage reported success while doing nothing — the same silent-success shape as the DT 400 and the masked | tee exit code. It now fails loudly, and the emptiness check runs on the unfiltered list so a libc-only binary doesn't false-FATAL.

Verified against live infrastructure

  • dhi Harbor endpoint: status=healthy, cred=webgrip — the Docker Hub credential authenticates dhi.io (same service="registry.docker.io"). No new account, no builder login.
  • dhi project public=true; anonymous manifest pull returns 200 — the builder needs no credential.
  • All three DHI digests cross-checked against the live proxy after pinning, to catch a runtime/-dev transposition.
  • grype resolves at /grype-src/usr/bin/grype, reports 0.116.1; jq-1.8.2 and yq v4.53.3 install from apk in the -dev variant; /bin/sh exists in the runtime variant.

Not verified

No successful build yet. The last unproven step is cve-gate reaching its own argument validation in the runtime assertion. If /out/lib is empty again it now fails at the staging step with the raw ldd output attached, rather than 40 relocation errors two stages later.

Merge order

  1. This PR → one image builds, cve-gate 0.1.0 lands in Harbor.
  2. Follow-up: restore the gate call site in .forgejo/actions/cosign-sign-attest (currently a commented block with its reasoning).
  3. Separately: the 17 OCI label repoints, batched.

ALLOW_MISSING_GATE exists as a loud, self-limiting bootstrap escape but should not be needed — the call site is deferred precisely so the happy path never touches it.

Refs VIK-631 (the techdocs GitHub-reference purge this deliberately leaves alone).

Ships the CVE budget gate and its dependencies. Cut from `main`, so it also carries the digest-pin commit that PR #71 missed. **Releases exactly one image: `cve-gate`.** No existing image directory is touched — verified: `git diff --name-only main...HEAD | grep '^ops/docker/'` returns only `ops/docker/cve-gate`. The OCI-label repoint that touches all 17 Dockerfiles is deliberately held back for a separate, batched PR, because `on_source_change` builds one matrix entry per changed dir and 17 concurrent builds would land on `fringe-workstation`. ## What's in it - **`ops/docker/cve-gate`** — the gate, and the worked example for ADR-0006. DHI `alpine-base` with the `-dev` variant as a *build stage only*; grype copied from `dhi.io/grype` instead of `curl | sh` of an install script from a mutable branch; non-root 65532; read-only rootfs; build-time assertions that run as the nonroot user. - **`ops/security/cve-budgets.yaml`** — per-image ceilings that ratchet. New images start in `warn`, the pipeline measures them, the budget is set at the observed number, then it only goes down. `ci-runner` (8C/136H) and `agent-runner` (1C/51H) seeded from measured Trivy Operator data. `cve-gate` is the only entry starting at `enforce` 0/0. - **`ops/vex/`** — OpenVEX as the only sanctioned suppression mechanism: hand-authored, PR-reviewed, justified from the closed vocabulary. Harbor's project-wide `cve_allowlist` stays empty. - **`.forgejo/actions/cve-gate`** — complete but **not yet called**. Wiring the call site in the same change that first builds the image would make `cve-gate`'s own release depend on an image that doesn't exist. Step 2 restores it. - **syft pinned to `cyclonedx-json@1.6`** — unpinned it emits 1.7; Dependency-Track tops out at 1.6 and returned `400` on every upload since run 133 (~50 releases), fail-soft, so nobody saw it. - **Dependency-Track failures split by class** — 5xx/000 (outage) stays a warning; 4xx (we sent something bad) becomes `::error::`. Loud, still non-fatal. - **`ops/kyverno/` deleted** — it targeted `ghcr.io/webgrip/*` with a Fulcio identity: a registry we no longer publish to, verified by a mechanism we no longer use. The live policy in `homelab-cluster` is correct; this copy was fiction that read as a guarantee. - **ADR-0004** supersedes ADR-0002 (keyless/GitHub → OpenBao Transit), stating the regressions plainly: no SLSA provenance, no transparency log. **ADR-0005** (VEX + budgets), **ADR-0006** (hardened bases, proposed). ## Build history — four real failures, all caught by assertions | Run | Failure | Fix | | --- | --- | --- | | 188 | `401` from dhi.io | `REGISTRY_DHI` defaults to the Harbor proxy | | 192 | jq relocation errors | copy jq's shared libraries, not just the binary | | 194 | same errors, `/out/lib` empty | musl's `ldd` writes to **stderr**; `2>/dev/null` discarded it | Run 188's real lesson: this repo's pattern is *default to the public registry, override to Harbor via `--build-arg`*. That is only safe because docker.io / ghcr.io / mcr.microsoft.com serve **anonymously** — a missing override just loses the cache. dhi.io refuses anonymous pulls, so the same pattern turns a missing flag into a hard failure. And it would keep being missing: `dockerVerifyGate()` in `@webgrip/semantic-release-config` hardcodes its own three `--build-arg` flags in another repo, and run 188 died in exactly that path. Inverting the default fixes both build paths at once. Run 194's real lesson: an empty result was accepted as an answer. "ldd returned nothing" and "this binary needs nothing" behaved identically, so the stage reported success while doing nothing — the same silent-success shape as the DT 400 and the masked `| tee` exit code. It now fails loudly, and the emptiness check runs on the *unfiltered* list so a libc-only binary doesn't false-FATAL. ## Verified against live infrastructure - `dhi` Harbor endpoint: `status=healthy`, `cred=webgrip` — the Docker Hub credential authenticates dhi.io (same `service="registry.docker.io"`). No new account, no builder login. - `dhi` project `public=true`; anonymous manifest pull returns `200` — the builder needs no credential. - All three DHI digests cross-checked against the live proxy after pinning, to catch a runtime/`-dev` transposition. - grype resolves at `/grype-src/usr/bin/grype`, reports `0.116.1`; `jq-1.8.2` and `yq v4.53.3` install from apk in the `-dev` variant; `/bin/sh` exists in the runtime variant. ## Not verified **No successful build yet.** The last unproven step is `cve-gate` reaching its own argument validation in the runtime assertion. If `/out/lib` is empty again it now fails at the staging step with the raw `ldd` output attached, rather than 40 relocation errors two stages later. ## Merge order 1. This PR → one image builds, `cve-gate 0.1.0` lands in Harbor. 2. Follow-up: restore the gate call site in `.forgejo/actions/cosign-sign-attest` (currently a commented block with its reasoning). 3. Separately: the 17 OCI label repoints, batched. `ALLOW_MISSING_GATE` exists as a loud, self-limiting bootstrap escape but should not be needed — the call site is deferred precisely so the happy path never touches it. Refs VIK-631 (the techdocs GitHub-reference purge this deliberately leaves alone).
fix(cve-gate): musl ldd writes to stderr — stop discarding it
All checks were successful
[Workflow] On Source Change / Guard: per-image .releaserc.cjs present (push) Successful in 2s
[Workflow] On Source Change / Determine Changed Directories (push) Successful in 19s
[Workflow] On Source Change / Determine changed images (push) Successful in 0s
[Workflow] On Source Change / Release (cve-gate, ops/docker/cve-gate) (push) Has been skipped
4e2ae224ac
Run 194 staged ZERO shared libraries and failed the runtime assertion with the
same jq relocation errors as run 192. The trace shows why:

    + ldd /usr/bin/jq
    + awk '...'
    + sort -u
    + read -r lib          <- one trace: the loop body never executed
    extra shared libraries staged for the runtime stage:
                           <- ls printed nothing

musl's ldd writes its dependency list to STDERR, not stdout. My `2>/dev/null`
discarded precisely the output the stage exists to read. `2>&1` instead.

The deeper bug was not the redirect, it was that an empty result was accepted as
an answer. "ldd returned nothing" and "this binary needs nothing" produced
identical behaviour, so the stage reported success while doing nothing at all —
the same silent-success shape as the Dependency-Track 400 and the `| tee` exit
code. It now distinguishes three outcomes explicitly:

  - "Not a valid dynamic program"  -> static, stage nothing, fine
  - paths resolved                 -> stage everything except libc and the loader
  - nothing at all                 -> FATAL, printing the raw ldd output

The emptiness check deliberately runs on the UNFILTERED list. Checking after
filtering would false-FATAL on a binary that legitimately needs only libc.

Also replaces the `while read` subshell with a plain `for` loop, so a failing
`install` aborts the build instead of being swallowed by the pipeline.

All four outcome shapes tested against real musl output formats before pushing:
dynamic-with-libs, static, dynamic-libc-only, and the run-194 empty case — which
now fails loudly at this stage rather than 40 relocation errors later.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
webgrip/infrastructure!74
No description provided.