No cargo test grades the Svelte extractor's rules — its fixtures run only in the release workflow, and three of its five rules are asserted nowhere #279
Labels
No labels
code-review
correctness
dos
performance
security
severity/high
severity/low
severity/medium
tech-debt
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
h-dv/code-index#279
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Found in a test-vacuity audit of
12bfe8f. This is the structural reason a boundary test could ship vacuous (fixed ine8d4694) and not be caught by any gate.Measured
1. The
.expectedfixtures are graded by nocargo test.tests/packages/svelte/fixtures/*.expectedare exercised only byplugin check "${DIGEST}"inside.forgejo/workflows/release.yml'spackage-plugin-sveltejob.ci.ymlnever runsplugin check; no test file undercrates/*/tests/invokes it on this package.Contrast
crates/daemon/tests/ruby_package_e2e.rs, which does exactly that forde.h-dv.ruby("all five fixtures must pass under the grant"). The Ruby package has the gate; the Svelte package does not.2.
crates/guest/svelteis outside the workspace (#276), socargo fmt --all,cargo clippy --workspaceand thedocsjob never reach the extractor source either.Together: none of the six verification gates CLAUDE.md lists grades the Svelte extractor's rules or its boundary. The audit lane had to replicate the release job by hand — pack, key, sign, trust, install, check — to grade them at all.
3. Three of the five rules have no assertion in any
cargo test.{#snippet}-> function symbolsvelte_package_e2e{@render row(x)}-> call ref<Header />-> type refonclick={h}-> call ref{a.b}-> read on the headwrite_corpusinsvelte_package_e2e.rscontains{@render legend(doubled)},onclick={handleClick}and{label}— and nothing asserted any ref from them untile8d4694added a ref-level reader for the boundary. The refs are produced; no cargo test looks at them.Why this matters more than the coverage number
The audit ran the mutation that disables the
<script>boundary entirely.plugin checkwent red on two fixtures — the fixtures work. The e2e test stayed green. So the one leg that caught the regression is the leg that runs only at release time, on a tag, after the commit has already landed on master.That is the wrong end of the loop for a gate whose whole job is to catch a silent behaviour change in a sandboxed extractor that ships to operators.
Related, same audit
GRAMMAR_BUILDSpresence is not enforced. Deleting the svelte row fromcrates/plugin-host/tests/grammar_provenance.rs'sGRAMMAR_BUILDSleaves all 7 tests green. The table has only!GRAMMAR_BUILDS.is_empty()— a typed-out list with no derivation and no floor.WASM_ARTIFACTSbeside it is derived from a repository walk and does catch a deleted row; this one does not. Same shaperelease_gate.rsdocuments as howde.h-dv.rubyonce shipped in no release.Ten recorded mutations carry a pointer instead of a verdict.
crates/abi/src/record.rsM-#229-a..d andcrates/indexer/src/packages.rsM-#229-e..j all read "RESULT: recorded at the run";svelte_package_e2e.rssaid "see the report on this change". The audit ran all ten and every one is RED, each killing exactly the test its doc names — so the tests are sound and only the record is thin.svelte_kind_table.rsis the counter-example done right: its verdicts are quoted inline and all reproduce, including the two survivors.Suggested shape
svelte_package_e2etest that runsplugin checkon the shipped package under the grant, mirroringruby_package_e2e. That is the single change that closes the largest gap.e8d4694adds arefs_inreader to that file; these are three asserts on top of machinery that now exists.GRAMMAR_BUILDS' population, or give it a floor, so a deleted row is red.Items 1 and 2 are the ones that would have caught the vacuous boundary test at
cargo testtime rather than at release time.Partial coverage improvement shipped in PR #432 / 0.41.15, with Svelte 0.2.1. Ordinary package tests now grade fixture expectations, including explicit absent decoy symbols/references and the negative Broken fixture; new cases cover legal text and attachments. The extractor is rebuilt from the package's own pinned sources, and the grammar rebuild applies a checksum-held source patch reproducibly. Actual published runtime acceptance also covers embedded JS/TS, attachment bindings, edits and restart. This does not complete the historical rule-by-rule mutation audit, so this issue remains open. Related remaining semantic gaps: #433 and #434.
Current-source audit on published0.41.21 narrows this old report. The largest pre-merge conformance gap is repaired: crates/first-party-tests/tests/svelte_package.rs::every_shipped_fixture_passes_runtime_conformance calls the shared production conform::check path and requires at least7 shipped fixtures. package-ci.yml runs package_leg.sh svelte release on scoped PRs/pushes, the same tier as package-release.yml; source qualification run6706 passed the Svelte package leg before this release. Grammar-population coverage is also repaired: grammar_provenance.rs::every_grammar_build_script_is_a_grammar_builds_row discovers the scripts and explicitly records the deleted-Svelte-row RED mutation. Remaining audit details are still real: the daemon's a_svelte_file_carries_symbols_through_the_real_daemon test directly asserts only ruleA, and several ABI/package mutation comments still say RESULT: recorded at the run. Keeping this issue open for those residual assertions/records; this is not evidence that #433/#434 are fixed.
Completed in PR #491 and released in v0.41.36.
Added a real-daemon Svelte template test for render calls, event-handler calls and member-head reads, including a negative assertion against the member tail. It has no script block, so another producer cannot rescue these assertions. Removing each of the three template constructs independently makes its corresponding assertion fail; the restored fixture passes.
Revalidated the existing shipped-fixture runtime conformance gate (all eight Svelte fixtures), derived grammar-build population fence and standalone Rust guest fmt/clippy/rustdoc gates. Removing the Svelte grammar registration fails the population test. The ten old mutation comments now attribute their RED outcomes to the original audit; this is not a claim that all ten mutations were rerun here.
The complete Svelte daemon suite (10 tests), shipped-fixture conformance and grammar provenance suite (7 tests) pass. Final Linux, Windows, package and pilot CI plus all ten release jobs are green. The installed published binaries passed signed-package upgrade tests and self-dogfood with eleven official packages over snapshot and daemon transports.