fix(cve-gate): chown the cache volumes; docker creates them root-owned #89

Merged
ryangr0 merged 1 commit from fix/cve-gate-volume-ownership into main 2026-08-05 18:15:47 +00:00
Owner

Run 238 turned no space left on device into permission denied. My fix for the first problem introduced the second.

unable to create listing temp file: open /var/tmp/cve-gate-scan/grype-db-listing...: permission denied
oci-registry: mkdir /var/tmp/cve-gate-scan/stereoscope-...: permission denied

Why

Docker creates a volume mount point as root:root 0755. The gate runs as uid 65532.

The tmpfs those volumes replaced had always worked because tmpfs mounts 1777. Swapping tmpfs → volume fixed the space problem and silently broke writability, and nothing in #87 tested that the container could still write to the paths it was being given.

Why not just fix it in the image

Docker copies ownership from the image's directory when a fresh volume is mounted over a path that already exists in the image. So the tidy fix is mkdir + chown in the cve-gate Dockerfile.

That route is closed: the gate that scans a new cve-gate release is the previous release. The image can never fix this for itself — a new cve-gate build would be gated by 0.3.1, which still has root-owned volumes. The action has to do it, and the gate image is dhi/static: no shell, no chown, nothing that can repair its own mount point from inside.

What this does

One short-lived helper container prepares the volumes before the gate runs — root, --cap-drop ALL, no-new-privileges, a shell and nothing else:

  • mkdir + chown 65532:65532 + chmod 0700 on both volumes
  • sweeps scratch older than two hours — stereoscope cleans up on a normal exit, but the scan volume is persistent now, so a killed run would otherwise leak a multi-GB extraction forever

Idempotent, so it costs one cached container start per gate run.

The scan volume also becomes named rather than anonymous. An anonymous volume is recreated — and re-rooted — every run, which would undo the chown each time.

The retry worked correctly

Three attempts, all exit 2, no signature, and the summary said NOT SIGNED rather than claiming one. It couldn't help here because the fault was local and permanent rather than a transient registry error. That's the intended behaviour, not a shortcoming — and it's why the exit contract only retries 2.

Honest note on the pattern

This is the third fix to the same twelve lines: docker cp into a read-only rootfs → 2 GB tmpfs → root-owned volumes. Each one was found by a release failing rather than by a test, because nothing exercises this action except a real release, and a real release takes ~25 minutes and mints an unsigned artifact when it fails.

If this run doesn't come back green, the next thing I'd build is a way to run the gate against an already-published digest outside the release path — so the container contract can be tested in a minute instead of a release cycle.

Run 238 turned `no space left on device` into `permission denied`. My fix for the first problem introduced the second. ``` unable to create listing temp file: open /var/tmp/cve-gate-scan/grype-db-listing...: permission denied oci-registry: mkdir /var/tmp/cve-gate-scan/stereoscope-...: permission denied ``` ## Why **Docker creates a volume mount point as `root:root 0755`. The gate runs as uid 65532.** The tmpfs those volumes replaced had always worked because **tmpfs mounts `1777`**. Swapping tmpfs → volume fixed the space problem and silently broke writability, and nothing in #87 tested that the container could still write to the paths it was being given. ## Why not just fix it in the image Docker copies ownership from the image's directory when a fresh volume is mounted over a path that already exists in the image. So the tidy fix is `mkdir` + `chown` in the cve-gate Dockerfile. That route is closed: **the gate that scans a new cve-gate release is the *previous* release.** The image can never fix this for itself — a new cve-gate build would be gated by 0.3.1, which still has root-owned volumes. The action has to do it, and the gate image is `dhi/static`: no shell, no `chown`, nothing that can repair its own mount point from inside. ## What this does One short-lived helper container prepares the volumes before the gate runs — root, `--cap-drop ALL`, `no-new-privileges`, a shell and nothing else: - `mkdir` + `chown 65532:65532` + `chmod 0700` on both volumes - sweeps scratch older than two hours — stereoscope cleans up on a normal exit, but the scan volume is **persistent** now, so a killed run would otherwise leak a multi-GB extraction forever Idempotent, so it costs one cached container start per gate run. The scan volume also becomes **named** rather than anonymous. An anonymous volume is recreated — and re-rooted — every run, which would undo the chown each time. ## The retry worked correctly Three attempts, all exit 2, no signature, and the summary said **NOT SIGNED** rather than claiming one. It couldn't help here because the fault was local and permanent rather than a transient registry error. That's the intended behaviour, not a shortcoming — and it's why the exit contract only retries 2. ## Honest note on the pattern This is the third fix to the same twelve lines: `docker cp` into a read-only rootfs → 2 GB tmpfs → root-owned volumes. Each one was found by a release failing rather than by a test, because **nothing exercises this action except a real release**, and a real release takes ~25 minutes and mints an unsigned artifact when it fails. If this run doesn't come back green, the next thing I'd build is a way to run the gate against an already-published digest outside the release path — so the container contract can be tested in a minute instead of a release cycle.
Run 238 replaced "no space left on device" with "permission denied":

    unable to create listing temp file: open /var/tmp/cve-gate-scan/grype-db-listing...: permission denied
    oci-registry: mkdir /var/tmp/cve-gate-scan/stereoscope-...: permission denied

Docker creates a volume mount point as root:root 0755. The gate runs as 65532. The tmpfs those
volumes replaced had always worked because tmpfs mounts 1777 — swapping to volumes fixed the space
problem and silently introduced an ownership one.

Docker copies ownership from the image's directory when a fresh volume is mounted over a path that
exists in the image, so the tidy fix is to create and chown those paths in the cve-gate Dockerfile.
That is not available here: the gate that scans a new cve-gate release is the PREVIOUS release, so
the image can never fix this for itself. The action has to, and the gate image is dhi/static — no
shell, no chown, nothing that can repair its own mount point from the inside.

So one short-lived helper container prepares them: root, --cap-drop ALL, no-new-privileges, a
shell and nothing else. It mkdirs, chowns to 65532, chmods 0700, and sweeps scratch older than two
hours — stereoscope cleans up on a normal exit, but the scan volume is persistent now, so a killed
run would otherwise leak a multi-GB extraction forever. Idempotent; costs one cached container
start per gate run.

The scan volume is also named rather than anonymous now. An anonymous volume is recreated (and
re-rooted) every run, which would undo the chown each time.

Retry behaviour is unchanged and did exactly what it should: three attempts, all exit 2, no
signature, and the summary said NOT SIGNED instead of claiming one. The retry could not help here
because the fault was local and permanent rather than a transient registry error — that is the
correct outcome, not a shortcoming.
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
webgrip/infrastructure!89
No description provided.