perf(docker-fast): drive the docker CLI directly instead of docker/*-action (2m04s -> ~9s setup) #42
No reviewers
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
webgrip/workflows!42
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "perf/fast-composite-cli"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
"Set up job" is 2m04s. The actual build is 21s.
What act is actually doing
Forgejo Actions runs on act (Forgejo Runner is a fork of Gitea's
act_runner, which embeds nektos/act). act does not fetch actions the way GitHub does.GitHub's runner downloads a tarball of a single ref from the API —
build-push-action@v6is ~1 MB and lands in under a second. actgit clones the action repository, full history, all refs, into a per-pod cache that dies with the pod.Measured on
infrastructurerun 224:docker/build-push-action@v6docker/login-action@v3docker/setup-buildx-action@v3docker/setup-qemu-action@v3actions/github-script@v8416 MB of git history to format command-line arguments. The actions aren't wasteful — act's fetch strategy is, and it isn't configurable. Same
uses:line costs ~1s on GitHub and 59s here.setup-qemu-actionis the clearest case: itsif:was false at the amd64 default and act fetched its 35 MB anyway, because everyuses:in a job is resolved before any condition is evaluated. It's now adocker run --privileged tonistiigi/binfmt --install allthat genuinely costs nothing when the condition is false.This isn't a new pattern here
@webgrip/semantic-release-config'sdockerVerifyGate()has been drivingdocker buildx create+docker buildx builddirectly in the release job all along, with no setup action anywhere.Verified against
infrastructurerun 220 before touching anything — buildx v0.34.1 at/usr/local/lib/docker/cli-plugins/docker-buildx, inherited from the ci-runner base image:The invocation here deliberately mirrors that one. There's also a
Verify docker toolchainstep that asserts both are present, because "the base image happens to ship buildx" is not something a workflow can see.Preserved
Tag normalization and registry enforcement, the
:cachetag, zstd-vs-gzip push expression,provenance=mode=max,sbom=true, and the build summary — now rendered in bash, which also drops a direct github.com dependency.Three things fixed on the way past
The buildkit image was an anonymous Docker Hub pull.
setup-buildx-actionwas given nodriver-opt, so every build pulledmoby/buildkitstraight from docker.io — rate-limited and outside Harbor. It now uses the Harbordockerhubproxy, whichdockerVerifyGate()was already doing.Inputs reached bash via
${{ }}interpolated into the script body. A tag containing a quote or backtick was pasted directly into bash source. They now arrive throughenv:.The buildkit container was never removed. It lives on the shared per-node dind daemon, not in the job pod, so it outlived every job.
setup-buildx-actioncleaned up in a post-step; composite actions have none, so it's an explicitif: always()step.One subtle flag detail
--provenance=mode=maxand--sbom=trueare single argv elements on purpose. buildx gives both aNoOptDefVal(so a bare--sbommeans--sbom=true). Passed as two arguments the flag takes its default and the value is parsed as a positional — silently becoming the build context.Verified locally against a docker stub
ghcr.io/evil/x:1)--push, no--output--output=type=image,…,push=true, no--push--targetset / unset--build-arg, noset -etripsha256:…from--metadata-file-, exit 0Expected effect
Release-path setup 124s → ~9s (just
actions/checkout, still 18 MB). That last one needs the mini-mirror work, not a rewrite.Blast radius
This composite is on the
-fastpath only, which callers opt into one at a time.docker-build-push-registry(the non-fast sibling) is untouched and keeps QEMU for genuine multi-arch.Measured on infrastructure run 224: "Set up job" 2m04s against 21s of actual build. act does not fetch actions the way GitHub does. GitHub's runner downloads a TARBALL of a single ref from the API — build-push-action@v6 is ~1MB and lands in under a second. act `git clone`s the action repository, full history, all refs, into a per-pod cache discarded with the pod: docker/build-push-action@v6 181 MB 58.6s docker/login-action@v3 134 MB 27.7s docker/setup-buildx-action@v3 66 MB 15.9s docker/setup-qemu-action@v3 35 MB 9.0s actions/github-script@v8 8 MB 2.9s 416 MB of git history to format command-line arguments. The actions are not wasteful; act's fetch strategy is, and it is not configurable. setup-qemu-action is the clearest case: its `if:` is false at the amd64 default, and act fetched its 35 MB anyway — every `uses:` in a job is resolved before any condition is evaluated. It is now a `docker run --privileged tonistiigi/binfmt --install all` that genuinely does nothing when the condition is false. This is not a new pattern here. @webgrip/semantic-release-config's dockerVerifyGate() has been driving `docker buildx create` + `docker buildx build` directly in the release job all along, with no setup action anywhere. Verified against infrastructure run 220: buildx v0.34.1 is present at /usr/local/lib/docker/cli-plugins/docker-buildx, inherited from the ci-runner base image. The invocation below mirrors the one already running in production — same driver, same buildkitd flags, same shape. Behaviour preserved: tag normalization, registry enforcement, the :cache tag, zstd-vs-gzip push expression, provenance=mode=max, sbom=true, and the build summary (now rendered in bash, which also removes a direct github.com dependency). Three things improved on the way past: - The buildkit image is now the Harbor dockerhub proxy rather than an anonymous docker.io pull. setup-buildx-action was given no driver-opt, so every build pulled moby/buildkit straight from Docker Hub — rate-limited and outside Harbor, which dockerVerifyGate already avoided. - Inputs reach bash through `env:` instead of `${{ }}` interpolated into the script body, so a tag containing a quote or backtick is no longer pasted into bash source. - The buildkit container is explicitly removed. It lives on the shared per-node dind daemon, not in the job pod, so it outlived every job; setup-buildx-action cleaned up in a post-step and composite actions have none. --provenance=mode=max and --sbom=true are single argv elements on purpose: buildx gives both a NoOptDefVal, so passed as two arguments the flag would take its default and the value would be parsed as a positional — silently becoming the build context. Verified locally against a docker stub: tag normalization (bare, fully-qualified, untagged, foreign-registry rejection), gzip --push vs zstd --output, target set and unset, build-args with spaces surviving as one argv element, empty build-args, digest extraction from --metadata-file, and summary rendering both after a successful build and when the build never ran.9462ad290ctoa6cd73ede4Amended: both images I introduced were floating tags, now digest-pinned. buildkit -> @sha256:2f5adac4ecd194d9f8c10b7b5d7bceb5186853db1b26e5abd3a657af0b7e26ec, binfmt -> @sha256:400a4873b838d1b89194d982c45e5fb3cda4593fbfd7e08a02e76b03b21166f0. binfmt:latest was the worse of the two: an unpinned mutable image run with --privileged, which is worse than the setup-qemu-action it replaced. buildx-stable-1 is a floating tag upstream repoints, so without the digest the thing bootstrapping every build in the org could change with no commit anywhere. Tag AND digest together, matching the convention in ops/docker/cve-gate/Dockerfile. The same unpinned buildx-stable-1 in infrastructure build-check is fixed in infrastructure #84.
a6cd73ede4to35f7fb2d4eCorrection: the PR body says the buildkit container was never removed because composite actions have no post-steps. That is wrong. act runs action post-steps and folds them into 'Complete job' instead of showing them as separate steps, which is why I missed them - infrastructure run 226 ends with 'Post Set up Docker Buildx' calling docker buildx rm and 'Post Log in to Harbor' calling docker logout. So the cleanup step here is the cleanup that already existed, made visible, not a leak being fixed. My version was also dropping the docker logout, which is restored now. Also renamed the step to 'Clean up builder and credentials' to reflect what it actually does.
forgejo-buildkitd is live (3/3 Ready, workers responding). Nothing was pointing at it. The docker-container driver starts a buildkit container, bootstraps it, builds, and is torn down — per job. Its LOCAL cache dies with it, so every build is locally cache-cold and leans entirely on `cache-from type=registry`: a network round-trip per layer against Harbor for what should be a disk read. The bootstrap time is the visible cost; the cold cache is the expensive one. Now: probe the daemon, use it when it answers, fall back otherwise. The fallback is load-bearing, not decoration — forgejo-buildkitd is deliberately NOT a dependency of the runner pool, so it can be absent, draining or not yet rolled out, and this composite runs in repos that may not have it at all. When it cannot be reached the build behaves exactly as it did before, slower, with a :⚠️: saying so. AMD64 ONLY on the remote path. `buildctl debug workers` reports linux/amd64 (+v2/v3/386): the daemon carries no emulation, and the binfmt step registers handlers through the dind daemon rather than inside buildkitd's container. Whether those kernel registrations are visible to it is unverified, so a non-amd64 request keeps the driver known to work instead of one that might. The entitlement flags do not cross over, and that is correct: --allow-insecure-entitlement configures a buildkitd this job starts, and forgejo-buildkitd is started without them. Checked before switching — no Dockerfile in webgrip/infrastructure uses `RUN --security=insecure` or `--network=host`. If one ever does it fails loudly on the remote path rather than silently building something different. Teardown now runs only for docker-container, which answers the objection that prompted this. That driver leaves a buildkit container on the shared per-node dind daemon, which outlives the pod — the only reason a teardown existed. The remote driver creates nothing remotely; its builder is a line in the ephemeral client's ~/.docker/buildx. `docker logout` still runs either way. Probing uses bash's /dev/tcp, exercising resolution and reachability in one go — the same technique the runner entrypoint already uses to pick its dind endpoint. The endpoint is a fixed DNS name rather than a per-pod node IP because the Service is internalTrafficPolicy: Local, so it resolves to this node's daemon.Extended: the composite now prefers forgejo-buildkitd, the shared per-node BuildKit daemon (confirmed live, 3/3 Ready, workers responding), and falls back to the throwaway docker-container builder when it does not answer. Remote path is AMD64 ONLY on purpose - buildctl debug workers reports linux/amd64 (+v2/v3/386), the daemon carries no emulation, and the binfmt step registers handlers through the dind daemon rather than inside buildkitd's container; whether those kernel registrations are visible to it is unverified, so a non-amd64 request keeps the driver known to work. The entitlement flags deliberately do not cross over: --allow-insecure-entitlement configures a buildkitd the job starts, and forgejo-buildkitd is started without them. Checked first - no Dockerfile in webgrip/infrastructure uses RUN --security=insecure or --network=host. Teardown now runs only for docker-container.