docs: say a Run's verification comes only from the worker's record #208

Merged

ryangr0 merged 2 commits from ryangr0/replace/204-verification-provenance into development 2026-10-04 08:39:22 +00:00 AGit

Conversation 0 Commits 2 Files changed 2 +2 -2

ryangr0 commented 2026-10-03 22:42:24 +00:00

Owner

Copy link

Replaces #204 (fix(ploeg): never read a current Run's verification from agent prose) after the Ploeg separation in #206. Its Ploeg part is ploeg-hq/ploeg#53; this PR moves Unfold's pin to that branch and carries the rest.

Stacked on the replacement for #195. Merge that one first; this PR then shows only its own two commits.

Commits

• 0f65ecf1 build(ploeg): pin ploeg-hq/ploeg#53 to keep verification provenance
• 1b288205 docs: say a Run's verification comes only from the worker's record (from 4d79debc13)

Merge order

1. #206, the cutover. Until it merges, this PR's diff also shows the cutover commits it is stacked on.
2. ploeg-hq/ploeg#53.
3. Move the pin in this PR's build(ploeg) commit to the merge commit on Ploeg's main, then merge. Until then the ploeg-pin check is red by design: it requires the pinned commit on Ploeg's main.

Verification (local, at 1b288205 with Ploeg e57d7e17)

• mise run verify with the result cache, as a pull request runs it: all gates passed. That includes the Ploeg group (its own scripts/verify.sh at the pin), Vloer, the extension, the demo replay check, integration (managed qualification) and docs.
• Ploeg side: GitHub CI passed on ploeg-hq/ploeg#53.

What happened to each commit of #204

Commit	Subject	Here
bb9b3861fa	fix(ploeg): retry a failed reviewer and close review_failed when no review came	comes from the #195 replacement this PR is stacked on
146cf69fee	fix(vloer): mark a pull request whose reviewer kept failing as unreviewed	comes from the #195 replacement this PR is stacked on
c2ced7378a	chore(site): re-record the demo replay with the review_failed wording	comes from the #195 replacement this PR is stacked on
4d79debc13	fix(ploeg): never read a current Run's verification from agent prose	Ploeg part ported to ploeg-hq/ploeg#53; Unfold part re-applied as 1b288205 ("docs: say a Run's verification comes only from the worker's record")

The original had no reviews or comments. Its checks were red because development itself failed at mise install --locked; #206 fixes that lock. No earlier check result carries over.

Original description of #204

Stacked on #195. It touches the same RoundReports select and RunReport fields. The three commits from #195 come first; this PR's own change is the last commit (4d79debc). Merge #195 first. Implements VIK-1780 (handoff EXEC-03), the remainder of VIK-1733 / #152.

Problem

Since #152, a structured verification record beats prose. But parseEvidence still treats every writing Run without a record as a Run from an older worker, and parses its summary marker [Ploeg verification passed] and its ### Ploeg verification section.

A current Run with no configured checks, or with checks skipped because it was cancelled or opened no pull request, also has no record. Its summary and findings are the agent's to write. So the usage report on the pull request showed Verification: passed and Commit verified: \``.

TestACurrentRunWithoutAWorkerRecordNeverShowsAPass reproduces this through the real store. On development it renders passed with the agent's invented commit.

Invariant

• For a Run ploegd stores now, the worker's structured record is the only verification evidence. Prose never supplies a result or a commit, and no record means "not recorded".
• Prose counts only for a Run stored before ploegd kept this distinction, and then only as visibly unverified history.
• A record that contradicts itself is refused at the API.

Change

• Migration 0036. It adds agent_runs.evidence_version (nullable SMALLINT, ≥ 1). ReportOutcome sets it to store.CurrentEvidenceVersion (1) for every reported outcome. Existing rows stay NULL; nothing infers provenance from their text.

• parseEvidence. A record always wins. With no record, a stamped Run is "not recorded". Only an unstamped Run goes through legacyEvidence, which now marks its result Historical.

• Usage report. A historical result reads Verification (historical prose, not a worker record): passed, unverified and Commit named in that prose (unverified): …, never Commit verified.

• Verification.Validate (applied by handleOutcome to every record; it refuses the report) now also refuses:

  • a passed record with zero checks;
  • a passed check with a nonzero exit code;
  • a failed check with exit code 0;
  • missing or out-of-order start and finish times;
  • a check that ran outside the record's time window;
  • a not_run check that carries an exit code or times.

  This is the chosen contract: these records are rejected, not normalised. Only a broken or forged worker sends one. It is documented in the schema description and on Validate.

• Docs. docs/how-to/review-an-agent-pr.md and the outcome schema describe "not recorded" and the historical label.

Acceptance criteria → tests

• Zero checks configured, spoofed marker, heading and hash → not recorded: TestACurrentRunWithoutAWorkerRecordNeverShowsAPass. It fails on development.
• Checks skipped by cancellation or a non-delivery outcome → same path, because no record means not recorded. The agent's own verification was already discarded by the worker (TestAnAgentCannotClaimTheWorkersVerification).
• A structured failure survives earlier or later narrative, code fences, duplicate markers and a different heading: TestAStructuredFailureSurvivesAnyNarrative.
• Contradictory records are rejected: 9 new cases in TestVerificationValidateRejectsInconsistentRecords, all of which validated on development, plus 2 API cases in TestValidateOutcomeReport.
• Old rows stay readable and labelled: TestProseOfARunStoredBeforeTheDistinctionIsUnverifiedHistory; the legacy fixture in TestUsageReportEvidenceParsesVerificationAndCommit now expects the historical label.
• Readers cannot store writing-run evidence: unchanged, covered by TestRoundReportsCarryTheWorkersVerification.

Test fixtures with zero timestamps were updated to real ones: the worker's incomplete record and the API's verification passed case.

Commands (at 4d79debc, Go 1.27.1, Node 24.21.0)

• mise exec -- go test ./pkg/harness/ ./pkg/shiftengine/ ./pkg/store/ ./pkg/httpapi/ ./pkg/worker/ -count=1 -v: 600 passed, 0 failed, 4 skipped (the operator qualification tests, which mise run integration runs)
• mise exec -- go test ./... -count=1 in apps/ploeg: 34 packages ok
• mise run verify: all gates passed (vloer 7, demo-replay 1, vloer-extension 4, ploeg 7, brand 1, site 5, site-demo 1, helm 15, release 3, integration 1, docs 1)

Migration and rollback

Additive column. An old binary ignores it. Rolling back the binary leaves the column in place and harmless. Migration number 0036: if another branch also adds 0036, renumber whichever merges second.

Not in scope

• Running checks on the exact pushed candidate (VIK-1738).
• Delivery identity (VIK-1732).

Refs VIK-1780

🤖 Generated with Claude Code

🤖 Generated with Claude Code

Replaces #204 (fix(ploeg): never read a current Run's verification from agent prose) after the Ploeg separation in #206. Its Ploeg part is [ploeg-hq/ploeg#53](https://github.com/ploeg-hq/ploeg/pull/53); this PR moves Unfold's pin to that branch and carries the rest. **Stacked on the replacement for #195.** Merge that one first; this PR then shows only its own two commits. ## Commits - `0f65ecf1` build(ploeg): pin ploeg-hq/ploeg#53 to keep verification provenance - `1b288205` docs: say a Run's verification comes only from the worker's record (from `4d79debc13`) ## Merge order 1. #206, the cutover. Until it merges, this PR's diff also shows the cutover commits it is stacked on. 2. [ploeg-hq/ploeg#53](https://github.com/ploeg-hq/ploeg/pull/53). 3. Move the pin in this PR's `build(ploeg)` commit to the merge commit on Ploeg's `main`, then merge. Until then the `ploeg-pin` check is red by design: it requires the pinned commit on Ploeg's `main`. ## Verification (local, at `1b288205` with Ploeg `e57d7e17`) - `mise run verify` with the result cache, as a pull request runs it: all gates passed. That includes the Ploeg group (its own `scripts/verify.sh` at the pin), Vloer, the extension, the demo replay check, integration (managed qualification) and docs. - Ploeg side: GitHub CI passed on [ploeg-hq/ploeg#53](https://github.com/ploeg-hq/ploeg/pull/53). ## What happened to each commit of #204 | Commit | Subject | Here | | --- | --- | --- | | `bb9b3861fa` | fix(ploeg): retry a failed reviewer and close review_failed when no review came | comes from the #195 replacement this PR is stacked on | | `146cf69fee` | fix(vloer): mark a pull request whose reviewer kept failing as unreviewed | comes from the #195 replacement this PR is stacked on | | `c2ced7378a` | chore(site): re-record the demo replay with the review_failed wording | comes from the #195 replacement this PR is stacked on | | `4d79debc13` | fix(ploeg): never read a current Run's verification from agent prose | Ploeg part ported to ploeg-hq/ploeg#53; Unfold part re-applied as `1b288205` ("docs: say a Run's verification comes only from the worker's record") | The original had no reviews or comments. Its checks were red because `development` itself failed at `mise install --locked`; #206 fixes that lock. No earlier check result carries over. <details><summary>Original description of #204</summary> **Stacked on #195.** It touches the same `RoundReports` select and `RunReport` fields. The three commits from #195 come first; this PR's own change is the last commit (`4d79debc`). Merge #195 first. Implements VIK-1780 (handoff EXEC-03), the remainder of VIK-1733 / #152. ## Problem Since #152, a structured verification record beats prose. But `parseEvidence` still treats every writing Run *without* a record as a Run from an older worker, and parses its summary marker `[Ploeg verification passed]` and its `### Ploeg verification` section. A current Run with no configured checks, or with checks skipped because it was cancelled or opened no pull request, also has no record. Its summary and findings are the agent's to write. So the usage report on the pull request showed `Verification: passed` and `Commit verified: \`<anything>\``. `TestACurrentRunWithoutAWorkerRecordNeverShowsAPass` reproduces this through the real store. On `development` it renders `passed` with the agent's invented commit. ## Invariant - For a Run ploegd stores now, the worker's structured record is the only verification evidence. Prose never supplies a result or a commit, and no record means "not recorded". - Prose counts only for a Run stored before ploegd kept this distinction, and then only as visibly unverified history. - A record that contradicts itself is refused at the API. ## Change - **Migration 0036.** It adds `agent_runs.evidence_version` (nullable `SMALLINT`, ≥ 1). `ReportOutcome` sets it to `store.CurrentEvidenceVersion` (1) for every reported outcome. Existing rows stay `NULL`; nothing infers provenance from their text. - **`parseEvidence`.** A record always wins. With no record, a stamped Run is "not recorded". Only an unstamped Run goes through `legacyEvidence`, which now marks its result `Historical`. - **Usage report.** A historical result reads `Verification (historical prose, not a worker record): passed, unverified` and `Commit named in that prose (unverified): …`, never `Commit verified`. - **`Verification.Validate`** (applied by `handleOutcome` to every record; it refuses the report) now also refuses: - a `passed` record with zero checks; - a passed check with a nonzero exit code; - a failed check with exit code 0; - missing or out-of-order start and finish times; - a check that ran outside the record's time window; - a `not_run` check that carries an exit code or times. This is the chosen contract: these records are **rejected**, not normalised. Only a broken or forged worker sends one. It is documented in the schema description and on `Validate`. - **Docs.** `docs/how-to/review-an-agent-pr.md` and the outcome schema describe "not recorded" and the historical label. ## Acceptance criteria → tests - **Zero checks configured, spoofed marker, heading and hash** → not recorded: `TestACurrentRunWithoutAWorkerRecordNeverShowsAPass`. It fails on `development`. - **Checks skipped by cancellation or a non-delivery outcome** → same path, because no record means not recorded. The agent's own verification was already discarded by the worker (`TestAnAgentCannotClaimTheWorkersVerification`). - **A structured failure survives earlier or later narrative, code fences, duplicate markers and a different heading**: `TestAStructuredFailureSurvivesAnyNarrative`. - **Contradictory records are rejected**: 9 new cases in `TestVerificationValidateRejectsInconsistentRecords`, all of which validated on `development`, plus 2 API cases in `TestValidateOutcomeReport`. - **Old rows stay readable and labelled**: `TestProseOfARunStoredBeforeTheDistinctionIsUnverifiedHistory`; the legacy fixture in `TestUsageReportEvidenceParsesVerificationAndCommit` now expects the historical label. - **Readers cannot store writing-run evidence**: unchanged, covered by `TestRoundReportsCarryTheWorkersVerification`. Test fixtures with zero timestamps were updated to real ones: the worker's `incomplete` record and the API's `verification passed` case. ## Commands (at 4d79debc, Go 1.27.1, Node 24.21.0) - `mise exec -- go test ./pkg/harness/ ./pkg/shiftengine/ ./pkg/store/ ./pkg/httpapi/ ./pkg/worker/ -count=1 -v`: 600 passed, 0 failed, 4 skipped (the operator qualification tests, which `mise run integration` runs) - `mise exec -- go test ./... -count=1` in apps/ploeg: 34 packages ok - `mise run verify`: all gates passed (vloer 7, demo-replay 1, vloer-extension 4, ploeg 7, brand 1, site 5, site-demo 1, helm 15, release 3, integration 1, docs 1) ## Migration and rollback Additive column. An old binary ignores it. Rolling back the binary leaves the column in place and harmless. Migration number 0036: if another branch also adds 0036, renumber whichever merges second. ## Not in scope - Running checks on the exact pushed candidate (VIK-1738). - Delivery identity (VIK-1732). Refs VIK-1780 🤖 Generated with [Claude Code](https://claude.com/claude-code) </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code)

ryangr0 added 11 commits 2026-10-03 22:42:24 +00:00

build: lock uv 0.12.22 in the root mise.lock ... 2cabad41a7

711a4814 moved uv to 0.12.22 in mise.toml but left the lockfile at
0.12.21. mise-action runs `mise install --locked`, which refuses a
version the lockfile does not hold, so the checks job has failed before
any gate since that commit, on development and on every pull request.

`mise lock uv` (mise 2026.9.18) records 0.12.22 for all seven platforms.
The openspec lock files it deletes under .mise/locks are kept.

Refs: https://github.com/webgrip/unfold/issues/2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

build(ploeg): consume Ploeg from ploeg-hq/ploeg as a submodule pinned at v0.1.0 ... 26a27b683d

Replace the vendored apps/ploeg tree with a gitlink to
https://github.com/ploeg-hq/ploeg.git at v0.1.0
(87f8dc45a0ea768c6ab95196d8b99d481df10c65), which was extracted from
this repository at 9c1d53f.

- mise run setup, the verify, docs and demo checkouts and the TechDocs
  prepare commands initialise the submodule.
- scripts/ploeg-pin.mjs refuses vendored source, another repository and
  an uninitialised or modified checkout. The ploeg-pin job also requires
  the pinned commit on Ploeg's main, and the release waits for it.
- verify runs Ploeg's own scripts/verify.sh at the pin and compiles the
  unified demo helper, which now imports github.com/ploeg-hq/ploeg.
- The docs build still renders Ploeg's pinned pages, but no longer
  regenerates or validates Ploeg's configuration reference, domain pages
  or decision ledger, and it links Ploeg's source files on GitHub at the
  pinned commit. The combined glossary keeps a decision that a pinned
  model cites by URL instead of mangling it into a relative path.
- Renovate ignores apps/ploeg, drops the Go overlay and leaves the pin
  to people. CI no longer builds the ploegd image context.

Refs: https://github.com/webgrip/unfold/issues/2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ci(release): stop versioning and publishing Ploeg from Unfold ... 52a7c89f35

Unfold's train now versions Vloer only; github.com/ploeg-hq/ploeg
versions and publishes Ploeg with GitHub Actions.

- on_release_published.yml drops the Ploeg chart, image, signing and
  distribution jobs. The Vloer publisher is the only one, so it takes
  the GitHub release out of draft itself.
- publish_release.py and publish_chart.py refuse ploeg before any Git,
  network or file access. The Go module export to github.com/webgrip/ploeg
  is gone, including the call that disabled GitHub Actions there.
- release-prepare.mjs and apps/.releaserc.cjs touch only Vloer's chart,
  and a commit scoped ploeg never releases Unfold.
- release-floors.json keeps Ploeg's floor and withdrawn 1.0.0-rc.1, marks
  the component retired after 0.4.0-rc.35, and both loaders refuse a
  train that versions a retired component.
- The release preflight no longer checks registry access for ploegd or
  charts/ploeg.

Refs: https://github.com/webgrip/unfold/issues/2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs: record that Unfold pins Ploeg and releases only Vloer ... 1bca2ac69b

ADR-0019 records the consumer side of the separation approved in
webgrip/unfold#1: Ploeg lives in github.com/ploeg-hq/ploeg, Unfold pins
it as a submodule, and the unfold-v train versions Vloer only. It
supersedes ADR-0004; ADR-0001 and ADR-0018 get dated notes.

README, AGENTS.md, NOTICE, the team-silver skill and the current pages
now say where Ploeg lives, how the pin moves, what Unfold releases and
where Ploeg's artifacts come from.

Refs: https://github.com/webgrip/unfold/issues/2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

build(docs): treat Ploeg's archived history pages as records ... 3870a8b560

Ploeg's main keeps its pre-separation release history in
docs/history/legacy-changelog.md, a record no current page links. Unfold
renders Ploeg's docs from the pinned commit, so any pin past v0.1.0 failed
the docs build with that page as an orphan. ploeg/history now joins
ploeg/backlog as a record path: kept, marked "not current guidance" and
left out of the nav and search.

Refs: https://github.com/webgrip/unfold/issues/2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

build(ploeg): pin ploeg-hq/ploeg#47 to retry a failed reviewer ... 42001ecc12

Moves apps/ploeg to 376ac60c, the head of ploeg-hq/ploeg#47, which is
the Ploeg side of Unfold PR 195. Once that pull request merges, move
the pin to its merge commit on Ploeg's main. Until then the ploeg-pin check
stays red, as it should.

Refs: https://github.com/ploeg-hq/ploeg/pull/47
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs: list review_failed in how to review an agent pull request ... 55c57ce42b

The Unfold side of "fix(ploeg): retry a failed reviewer and close
review_failed when no review came" (ploeg-hq/ploeg#47): the review guide
names the new close reason and says it still settles a delivered pull
request as awaiting_review.

Refs VIK-1304

Replaces-commit: bb9b3861fa (#195)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

fix(vloer): mark a pull request whose reviewer kept failing as unreviewed ... d58ff0ac0c

Ploeg now closes a Shift review_failed when its reviewer Run failed and
its retries ran out. The pull request still waits for review, so the
item sits in Ready for review, where its row read "No agent verdict".

The review lane chip now reads "Agent review unavailable" with the
reason in its title, the Work Item page says no agent reviewed the pull
request, and the close reason reads "No agent reviewed it: the reviewer
kept failing" wherever Shift close reasons are listed.

VIK-1304

Replaces-commit: 146cf69fee (#195)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

chore(site): re-record the demo replay with the review_failed wording ... a4dabce0df

VIK-1304

Re-recorded with `mise run demo-record` on top of the Ploeg separation
(webgrip/unfold#206) instead of merging the recorded JSON.

Replaces-commit: c2ced7378a (#195)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

build(ploeg): pin ploeg-hq/ploeg#53 to keep verification provenance ... 0f65ecf147

Moves apps/ploeg to e57d7e17, the head of ploeg-hq/ploeg#53, which is
the Ploeg side of Unfold PR 204. Once that pull request merges, move
the pin to its merge commit on Ploeg's main. Until then the ploeg-pin check
stays red, as it should.

Refs: https://github.com/ploeg-hq/ploeg/pull/53
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs: say a Run's verification comes only from the worker's record ... 1b28820551

The Unfold side of "fix(ploeg): never read a current Run's verification
from agent prose" (ploeg-hq/ploeg#53): the review guide says a missing
record reads "not recorded", including checks skipped because the Run was
cancelled or opened no pull request, and that only Runs stored before
VIK-1780 show their verification prose, labelled historical and unverified.

Refs VIK-1780

Replaces-commit: 4d79debc13 (#204)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

ryangr0 force-pushed ryangr0/replace/204-verification-provenance from 1b28820551 to e8ec7ba335 2026-10-04 07:08:53 +00:00 Compare

ryangr0 force-pushed ryangr0/replace/204-verification-provenance from e8ec7ba335 to 8c32a31bbd 2026-10-04 07:12:27 +00:00 Compare

ryangr0 merged commit 60596e1771 into development 2026-10-04 08:39:22 +00:00

ryangr0 referenced this pull request from a commit 2026-10-04 08:39:25 +00:00

Merge pull request 'docs: say a Run's verification comes only from the worker's record' (#208) from ryangr0/replace/204-verification-provenance into development

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

webgrip/unfold!208

No description provided.