v0.27.0 invalidates every stored conformance verdict — needs a release note, and the wording invites a wrong diagnosis #211

New issue

Closed

opened 2026-09-07 16:57:09 +02:00 by buildagent · 0 comments

buildagent commented

2026-09-07 16:57:09 +02:00

Member

Found by dogfooding the v0.27.0 release candidate (caa62fe). Not a code defect — the classification is correct and conservative. It is an upgrade consequence nobody has named, plus a wording problem.

What an operator sees after upgrading

[ WARN ] conformance verdicts   0 passed, 0 not run, 0 FAILED,
                                1 passed under another engine
                                (sha256:584fe718… (passed under `wasmtime 36.0.14`,
                                 this host is `wasmtime 36.0.14; epoch=1;fuel=1;
                                 stack=1048576;rsimd=0;rsimd_det=1;multimem=0;backtrace=0`))
   Repair: run `code-index plugin check <digest>` for each …

The wasmtime version is identical on both sides: 36.0.14. What changed is that the engine IDENTITY STRING gained a knob suffix.

Why every installation is affected

bff9f15 (2026-09-04) added the knob suffix to the identity format! in crates/plugin-host/src/engine.rs:

+  "epoch={};fuel={};stack={};rsimd={};rsimd_det={};multimem={};backtrace={}"

git merge-base --is-ancestor bff9f15 v0.26.1 → false. The commit is unreleased and ships first in v0.27.0.

So on upgrade, engine_is_current() returns false for every verdict recorded by any previous release, on every machine. Each one lands in the stale bucket, and plugin doctor WARNs until the operator re-runs code-index plugin check once per installed package.

The behaviour is right; do not "fix" it

crates/cli/src/plugin_doctor.rs:651 is doing the correct thing. The knobs (epoch, fuel, stack, rsimd_det, …) genuinely change execution semantics, so a C1 verdict recorded without knowing them is not provably transferable to this host. Treating an unknown-knob verdict as still-passing would be exactly the "measured under different conditions, reported as measured here" collapse this tree refuses everywhere else. Widening engine_is_current() to ignore the suffix would be a regression in honesty.

What actually needs doing

1. Release note for v0.27.0. State plainly that this release changes the recorded engine identity, that existing conformance verdicts therefore read as stale, that plugin doctor will WARN, and that the one-line remedy is code-index plugin check <digest> per installed package. An operator who upgrades and meets an unexplained WARN on a package that was fine yesterday will otherwise assume the upgrade broke their plugin.

2. Wording. "passed under another engine" is literally true of the identity record but reads as "a different wasmtime", sending the reader to hunt for an engine change that did not happen. When the two strings share a version prefix and differ only by added fields, say so — e.g. "recorded before this host began pinning engine knobs (same wasmtime 36.0.14); the verdict predates the fields, it was not taken under a different engine". The distinction is between a different engine and a less precise record of the same one, and only the second is what happened here.

That wording change is worth a test: a verdict whose engine string is a strict prefix of the host's should render as the predates-the-fields case, and one with a genuinely different version should still render as another engine. Mutate the prefix test to == and the first assertion must go red.

Not affected

Shipped first-party packages are unaffected on a fresh install: a new user has no prior verdict, so they land in not_run, which already reads correctly. This only bites installations that carry verdicts recorded by an earlier release.

Found by dogfooding the v0.27.0 release candidate (`caa62fe`). **Not a code defect** — the classification is correct and conservative. It is an upgrade consequence nobody has named, plus a wording problem. ## What an operator sees after upgrading ``` [ WARN ] conformance verdicts 0 passed, 0 not run, 0 FAILED, 1 passed under another engine (sha256:584fe718… (passed under `wasmtime 36.0.14`, this host is `wasmtime 36.0.14; epoch=1;fuel=1; stack=1048576;rsimd=0;rsimd_det=1;multimem=0;backtrace=0`)) Repair: run `code-index plugin check <digest>` for each … ``` The wasmtime version is **identical on both sides**: 36.0.14. What changed is that the engine IDENTITY STRING gained a knob suffix. ## Why every installation is affected `bff9f15` (2026-09-04) added the knob suffix to the identity `format!` in `crates/plugin-host/src/engine.rs`: ``` + "epoch={};fuel={};stack={};rsimd={};rsimd_det={};multimem={};backtrace={}" ``` `git merge-base --is-ancestor bff9f15 v0.26.1` → **false**. The commit is unreleased and ships first in v0.27.0. So on upgrade, `engine_is_current()` returns false for **every verdict recorded by any previous release**, on every machine. Each one lands in the `stale` bucket, and `plugin doctor` WARNs until the operator re-runs `code-index plugin check` once per installed package. ## The behaviour is right; do not "fix" it `crates/cli/src/plugin_doctor.rs:651` is doing the correct thing. The knobs (`epoch`, `fuel`, `stack`, `rsimd_det`, …) genuinely change execution semantics, so a C1 verdict recorded without knowing them is not provably transferable to this host. Treating an unknown-knob verdict as still-passing would be exactly the "measured under different conditions, reported as measured here" collapse this tree refuses everywhere else. Widening `engine_is_current()` to ignore the suffix would be a regression in honesty. ## What actually needs doing **1. Release note for v0.27.0.** State plainly that this release changes the recorded engine identity, that existing conformance verdicts therefore read as stale, that `plugin doctor` will WARN, and that the one-line remedy is `code-index plugin check <digest>` per installed package. An operator who upgrades and meets an unexplained WARN on a package that was fine yesterday will otherwise assume the upgrade broke their plugin. **2. Wording.** "passed under another engine" is literally true of the identity record but reads as *"a different wasmtime"*, sending the reader to hunt for an engine change that did not happen. When the two strings share a version prefix and differ only by added fields, say so — e.g. *"recorded before this host began pinning engine knobs (same `wasmtime 36.0.14`); the verdict predates the fields, it was not taken under a different engine"*. The distinction is between **a different engine** and **a less precise record of the same one**, and only the second is what happened here. That wording change is worth a test: a verdict whose engine string is a strict prefix of the host's should render as the predates-the-fields case, and one with a genuinely different version should still render as another engine. Mutate the prefix test to `==` and the first assertion must go red. ## Not affected Shipped first-party packages are unaffected on a fresh install: a new user has no prior verdict, so they land in `not_run`, which already reads correctly. This only bites installations that carry verdicts recorded by an earlier release.

buildagent referenced this issue

2026-09-07 17:58:36 +02:00

evidence_gaps.semantics tells the reader to consult partial_sources, and no tool ever emits that field #216

buildagent referenced this issue

2026-09-07 17:58:59 +02:00

plugin status and the operator guide still call a predates-the-knobs verdict "another engine" — drift introduced by #211's fix #217