h-dv/code-index
Watch
4
0
Fork
You've already forked code-index
 0
Code Issues 46 Pull requests Projects Releases 98 Packages Wiki Activity Actions 1

test: remaining corpus metamorphic gates — permutation, project split and plugin generations #42

New issue
Open
opened 2026-07-28 20:38:18 +02:00 by buildagent · 2 comments
buildagent commented 2026-07-28 20:38:18 +02:00
Member
Copy link

Current state

The corpus metamorphic suite already ships id-independent comparisons for:

  • cold determinism;
  • path invariance;
  • cold vs incremental edit;
  • cold vs incremental delete;
  • cold vs incremental rename.

It found #52’s sticky-resolution defect. #52 is now closed, so the temporary known-incoherence allowance must be absent or removed as part of this issue’s next verification.

The original unimplemented invariants remain:

  • file-walk permutation invariance;
  • single-project vs primary+link split invariance.

#75 adds package/generation invariants that belong in the same real-corpus gate.

Remaining goal

Prove that irrelevant ordering, absolute location, project routing and plugin-generation construction do not change the canonical active semantic projection.

All comparisons render targets by stable semantic identity such as path/name/span/package generation, never SQLite ids.

Required invariants

Existing-suite cleanup

  • Verify #52’s allowlist is empty and fails if reintroduced unnecessarily.
  • Keep first-difference reporting and positive controls.
  • Preserve separate edit/delete/rename tests.

Walk permutation

Add an explicit deterministic walker-order test seam. Index each corpus under multiple committed seeds/orders.

Assert identical:

  • file contributions;
  • symbols/refs/imports;
  • resolved targets/resolved_by/influence;
  • aggregates and symbol edges;
  • plugin coverage/disclosures.

The seam is test-only and cannot affect production default order.

Project split

Compare the same source tree represented as:

  1. one project;
  2. primary plus linked subprojects with equivalent active package sets.

Do not demand cross-project edges that the product explicitly does not support. Instead define and compare the per-project projections and fan-out merge contract. Assert no fabricated cross-project target.

Also test differing package sets per linked project; no process-global plugin registry may leak eligibility or language profiles across them.

Plugin generation construction

For exact activation identity G compare:

  • cold index directly under G;
  • cold under previous generation then activate G;
  • watcher convergence after enabling G;
  • rollback to the retained previous generation.

Use XAML and the complete migrated-language package from #80. Include package-only changes with unchanged source bytes, precedence swaps and embedded-language files.

Encoding/hostile content

Retain fixture-scale encoding guards and add corpus mutations for CRLF/BOM and one content-refused path. Permanent refusal must be disclosed consistently rather than compared as an empty index.

Positive controls

Every run proves:

  • at least one repo executed;
  • index/fact populations are non-empty;
  • permutation actually changes order;
  • split actually changes topology;
  • package change changes expected package-produced facts;
  • rollback changes active generation;
  • mutation changes source or semantic identity as intended.

Acceptance

  1. Existing five suites are green with no stale #52 allowance.
  2. At least three distinct walk permutations produce identical projections on all tier-1 corpora.
  3. Single-vs-linked topology obeys a documented comparison contract with no cross-project fabrication.
  4. Cold, activated, watcher-converged and rolled-back plugin generations agree where semantically equivalent.
  5. Linked projects can carry different plugin sets without eligibility/profile leakage.
  6. Every assertion registers a positive control and reports its first differing row.
## Current state The corpus metamorphic suite already ships id-independent comparisons for: - cold determinism; - path invariance; - cold vs incremental edit; - cold vs incremental delete; - cold vs incremental rename. It found #52’s sticky-resolution defect. #52 is now closed, so the temporary known-incoherence allowance must be absent or removed as part of this issue’s next verification. The original unimplemented invariants remain: - file-walk permutation invariance; - single-project vs primary+link split invariance. #75 adds package/generation invariants that belong in the same real-corpus gate. ## Remaining goal Prove that irrelevant ordering, absolute location, project routing and plugin-generation construction do not change the canonical active semantic projection. All comparisons render targets by stable semantic identity such as path/name/span/package generation, never SQLite ids. ## Required invariants ### Existing-suite cleanup - Verify #52’s allowlist is empty and fails if reintroduced unnecessarily. - Keep first-difference reporting and positive controls. - Preserve separate edit/delete/rename tests. ### Walk permutation Add an explicit deterministic walker-order test seam. Index each corpus under multiple committed seeds/orders. Assert identical: - file contributions; - symbols/refs/imports; - resolved targets/resolved_by/influence; - aggregates and symbol edges; - plugin coverage/disclosures. The seam is test-only and cannot affect production default order. ### Project split Compare the same source tree represented as: 1. one project; 2. primary plus linked subprojects with equivalent active package sets. Do not demand cross-project edges that the product explicitly does not support. Instead define and compare the per-project projections and fan-out merge contract. Assert no fabricated cross-project target. Also test differing package sets per linked project; no process-global plugin registry may leak eligibility or language profiles across them. ### Plugin generation construction For exact activation identity G compare: - cold index directly under G; - cold under previous generation then activate G; - watcher convergence after enabling G; - rollback to the retained previous generation. Use XAML and the complete migrated-language package from #80. Include package-only changes with unchanged source bytes, precedence swaps and embedded-language files. ### Encoding/hostile content Retain fixture-scale encoding guards and add corpus mutations for CRLF/BOM and one content-refused path. Permanent refusal must be disclosed consistently rather than compared as an empty index. ## Positive controls Every run proves: - at least one repo executed; - index/fact populations are non-empty; - permutation actually changes order; - split actually changes topology; - package change changes expected package-produced facts; - rollback changes active generation; - mutation changes source or semantic identity as intended. ## Acceptance 1. Existing five suites are green with no stale #52 allowance. 2. At least three distinct walk permutations produce identical projections on all tier-1 corpora. 3. Single-vs-linked topology obeys a documented comparison contract with no cross-project fabrication. 4. Cold, activated, watcher-converged and rolled-back plugin generations agree where semantically equivalent. 5. Linked projects can carry different plugin sets without eligibility/profile leakage. 6. Every assertion registers a positive control and reports its first differing row.
buildagent commented 2026-07-28 21:36:48 +02:00
Author
Member
Copy link

Landed — crates/indexer/tests/corpus_metamorphic.rs

Five asserting tests over all 7 tier-1 repos, all comparing on the id-independent projection (a ref's target rendered path#name@line, never target_id):

test result
corpus_determinism 7/7 identical
corpus_path_invariance 7/7 identical (/s vs /d/e/e/p/e/r/still)
corpus_cold_equals_incremental_edit 7/7 identical
corpus_cold_equals_incremental_delete 7/7 identical
corpus_cold_equals_incremental_rename 6/7 clean, cs-dapper diverges → #52

The three mutation kinds are split one per test rather than applied together, so a failure names the responsible operation instead of leaving three suspects. That split is what localised #52.

It found a real defect: #52

The combined test failed on cs-dapper. Isolating the three mutations showed the rename alone was responsible: renaming Dapper/SqlMapper.cs — semantically a no-op in C# — leaves 3 refs in CommandDefinition.cs resolved to SqlMapper.Settings.cs that a cold index of the identical tree declines to resolve. Root-caused to sticky resolution (target_id IS NULL gating) plus name-only invalidation. Filed as #52, deliberately not fixed here.

Those 3 sites are enumerated in known_incoherence() with a written reason and an issue reference — the oracle.toml name_fallback_ceiling device. assert_same_except tolerates only those exact (path, name, kind, line, col) sites; symbol differences are never tolerated, and any other ref difference still fails. Verified non-vacuous: the run reports 6 known-incoherence ref difference(s) tolerated (3 sites × both sides of the symmetric difference), so a stale list cannot pass silently. The list must shrink to empty when #52 lands.

Deviation from the issue's plan

Permutation invariance is NOT implemented. The walk order isn't externally controllable, so there is no honest way to test it without a walker seed knob. I'd rather say so than ship something that looks like the invariant but isn't. Split invariance (single project vs primary+link) is also not covered yet. Both remain open on this issue.

Path invariance — the one invariant here the spike never tested — is clean on all 7.

Acceptance: boxes 2, 3, 4, 5 met; box 1 partially (4 of 6 invariants).

## Landed — `crates/indexer/tests/corpus_metamorphic.rs` Five asserting tests over all 7 tier-1 repos, all comparing on the **id-independent projection** (a ref's target rendered `path#name@line`, never `target_id`): | test | result | |---|---| | `corpus_determinism` | 7/7 identical | | `corpus_path_invariance` | 7/7 identical (`/s` vs `/d/e/e/p/e/r/still`) | | `corpus_cold_equals_incremental_edit` | 7/7 identical | | `corpus_cold_equals_incremental_delete` | 7/7 identical | | `corpus_cold_equals_incremental_rename` | 6/7 clean, **cs-dapper diverges → #52** | The three mutation kinds are **split one per test** rather than applied together, so a failure names the responsible operation instead of leaving three suspects. That split is what localised #52. ## It found a real defect: #52 The combined test failed on cs-dapper. Isolating the three mutations showed the rename alone was responsible: renaming `Dapper/SqlMapper.cs` — semantically a no-op in C# — leaves 3 refs in `CommandDefinition.cs` resolved to `SqlMapper.Settings.cs` that a cold index of the identical tree declines to resolve. Root-caused to sticky resolution (`target_id IS NULL` gating) plus name-only invalidation. Filed as **#52**, deliberately not fixed here. Those 3 sites are enumerated in `known_incoherence()` with a written reason and an issue reference — the `oracle.toml` `name_fallback_ceiling` device. `assert_same_except` tolerates **only** those exact `(path, name, kind, line, col)` sites; symbol differences are never tolerated, and any other ref difference still fails. Verified non-vacuous: the run reports `6 known-incoherence ref difference(s) tolerated` (3 sites × both sides of the symmetric difference), so a stale list cannot pass silently. **The list must shrink to empty when #52 lands.** ## Deviation from the issue's plan **Permutation invariance is NOT implemented.** The walk order isn't externally controllable, so there is no honest way to test it without a walker seed knob. I'd rather say so than ship something that looks like the invariant but isn't. Split invariance (single project vs primary+link) is also not covered yet. Both remain open on this issue. Path invariance — the one invariant here the spike never tested — is clean on all 7. Acceptance: boxes 2, 3, 4, 5 met; box 1 partially (4 of 6 invariants).
buildagent changed title from test: corpus metamorphic gate — determinism, cold==incremental, permutation and path invariance on real repos to test: remaining corpus metamorphic gates — permutation, project split and plugin generations 2026-08-26 13:40:26 +02:00
buildagent commented 2026-09-07 00:14:47 +02:00
Author
Member
Copy link

Acceptance clause 1 is now GRADED, not just true. The rest of the issue is untouched.

Branch worktree-agent-a9fb463736bf2b59d, based on master 1d81180. Not pushed.

What I found

known_incoherence in crates/indexer/tests/corpus_metamorphic.rs already returns vec![], and its doc already says "Empty, and it must stay empty… do not add entries to make a red run green." So the first half of clause 1 ("verify the allowlist is empty") held.

The second half — "and fails if reintroduced unnecessarily" — was enforced by nothing. Prose is not a gate. And the shape of the hole is this project's own: the suites that CONSUME the allowance are corpus suites, so without COSI_CORPUS_DIR they report executed=0 unavailable=1 and pass. An entry could have been added, reviewed, merged and never once executed.

What I added

the_known_incoherence_allowance_is_empty — a sweep over the tier-1 roster requiring the allowance to be empty for every repo. It needs no corpus: corpus::tier reads the checked-in manifest, so it runs on every push and an entry appears the moment it is written.

It is a waiver gate, not a ban. An entry there is a documented product defect and there may one day be a right one; adding it now has to be a decision recorded next to the emptiness it breaks, rather than a quiet accommodation inside a suite nobody runs locally. Same shape as ignored_test_reachability.

Two mutations, both run:

known_incoherence returns one entry
  RED — rust-ripgrep: ("a.rs", "f", "call", 1, 2) — #52 came back
        ts-zod: …  php-guzzle: …  ruby-sinatra: …  cs-dapper: …   (one line per repo)

roster read from corpus::tier(99), i.e. empty
  RED — the tier-1 roster holds 0 repo(s), which is too few to have read
        tests/corpus/corpus.toml — this sweep would pass over nothing

The second is the one that matters: a sweep over an empty roster passes, so the floor is what stops this test from being the very thing it exists to prevent.

Explicitly NOT done — state of the rest of this issue on 1d81180

corpus_metamorphic.rs today ships determinism, cold-vs-incremental edit / delete / add / rename, and path invariance (7 tests). Of the remaining goal:

  • walk permutation invariance — no seam, no test. grep for a walker-order seam finds only generation_equivalence.rs's influence-row permutation, which is a different thing.
  • project split (single vs primary+linked) — nothing.
  • plugin generation construction (cold under G / activate G / watcher-converged / rollback) — nothing here; generation_equivalence.rs covers adjacent ground at fixture scale, not on the corpus.
  • CRLF/BOM and content-refused corpus mutations — nothing.

Those are the substantive remainder and I did not start them; this comment is only so the first acceptance clause stops being a promise.

## Acceptance clause 1 is now GRADED, not just true. The rest of the issue is untouched. Branch `worktree-agent-a9fb463736bf2b59d`, based on master `1d81180`. Not pushed. ### What I found `known_incoherence` in `crates/indexer/tests/corpus_metamorphic.rs` already returns `vec![]`, and its doc already says *"Empty, and it must stay empty… do not add entries to make a red run green."* So the first half of clause 1 ("verify the allowlist is empty") held. **The second half — "and fails if reintroduced unnecessarily" — was enforced by nothing.** Prose is not a gate. And the shape of the hole is this project's own: the suites that CONSUME the allowance are corpus suites, so without `COSI_CORPUS_DIR` they report `executed=0 unavailable=1` and **pass**. An entry could have been added, reviewed, merged and never once executed. ### What I added `the_known_incoherence_allowance_is_empty` — a sweep over the tier-1 roster requiring the allowance to be empty for every repo. It needs **no corpus**: `corpus::tier` reads the checked-in manifest, so it runs on every push and an entry appears the moment it is written. It is a **waiver gate, not a ban**. An entry there is a documented product defect and there may one day be a right one; adding it now has to be a decision recorded next to the emptiness it breaks, rather than a quiet accommodation inside a suite nobody runs locally. Same shape as `ignored_test_reachability`. Two mutations, both run: ``` known_incoherence returns one entry RED — rust-ripgrep: ("a.rs", "f", "call", 1, 2) — #52 came back ts-zod: … php-guzzle: … ruby-sinatra: … cs-dapper: … (one line per repo) roster read from corpus::tier(99), i.e. empty RED — the tier-1 roster holds 0 repo(s), which is too few to have read tests/corpus/corpus.toml — this sweep would pass over nothing ``` The second is the one that matters: a sweep over an empty roster passes, so the floor is what stops this test from being the very thing it exists to prevent. ### Explicitly NOT done — state of the rest of this issue on `1d81180` `corpus_metamorphic.rs` today ships determinism, cold-vs-incremental **edit / delete / add / rename**, and **path invariance** (7 tests). Of the remaining goal: - **walk permutation invariance** — no seam, no test. `grep` for a walker-order seam finds only `generation_equivalence.rs`'s influence-row permutation, which is a different thing. - **project split (single vs primary+linked)** — nothing. - **plugin generation construction** (cold under G / activate G / watcher-converged / rollback) — nothing here; `generation_equivalence.rs` covers adjacent ground at fixture scale, not on the corpus. - **CRLF/BOM and content-refused corpus mutations** — nothing. Those are the substantive remainder and I did not start them; this comment is only so the first acceptance clause stops being a promise.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
p1-integration
p0-integration
release/v0.32.2
release/v0.32.1
epic-i072-release-032
windows-installer
i269b-registry-truth
probe-win-baseline
fix/246-shadowed-imports
release/0.28.1
fix/review-followthrough-20260911
iwin-msvc-archive
v0.32.2
v0.32.1
v0.32.0
v0.31.2
v0.31.1
v0.31.0
v0.30.1
v0.30.0
v0.29.0
v0.28.3
v0.28.2
v0.28.1
v0.28.0
v0.27.1
v0.27.0
v0.26.1
v0.26.0
v0.25.0
v0.24.1
v0.24.0
v0.23.1
v0.23.0
v0.22.3
v0.22.2
v0.22.1
v0.22.0
v0.21.0
v0.20.3
v0.20.2
v0.20.1
v0.20.0
v0.19.5
v0.19.4
v0.19.3
v0.19.2
v0.19.1
v0.19.0
v0.18.1
v0.18.0
v0.17.0
v0.16.0
v0.15.3
v0.14.0
v0.13.3
v0.13.2
v0.13.1
v0.13.0
v0.12.0
v0.11.7
v0.11.6
v0.11.5
v0.11.4
v0.11.3
v0.11.2
v0.11.1
v0.11.0
v0.10.2
v0.10.1
v0.10.0
v0.9.0
v0.8.7
v0.8.6
v0.8.5
v0.8.4
v0.8.3
v0.8.2
v0.8.1
v0.8.0
v0.7.1
v0.7.0
v0.6.1
v0.6.0
v0.5.21
v0.5.20
v0.5.19
v0.5.18
v0.5.17
v0.5.16
v0.5.15
v0.5.14
v0.5.13
v0.5.12
v0.5.11
v0.5.10
v0.5.9
v0.5.8
v0.5.7
v0.5.6
v0.5.5
v0.5.4
v0.5.3
v0.5.2
v0.5.1
v0.5.0
v0.4.2
v0.4.1
v0.4.0
v0.3.3
v0.3.2
v0.3.1
v0.3.0
v0.2.0
Labels
Clear labels   
code-review

   
correctness

   
dos

   
performance

   
security

   
severity/high

   
severity/low

   
severity/medium

   
tech-debt
  
Kind/Breaking
Breaking change that won't be backward compatible

  
Kind/Bug
Something is not working

  
Kind/Documentation
Documentation changes

  
Kind/Enhancement
Improve existing functionality

  
Kind/Feature
New functionality

  
Kind/Security
This is security issue

  
Kind/Testing
Issue or pull request related to testing

  
Priority
Critical
 The priority is critical

  
Priority
High
 The priority is high

  
Priority
Low
 The priority is low

  
Priority
Medium
 The priority is medium

  
Reviewed
Confirmed
 Issue has been confirmed

  
Reviewed
Duplicate
 This issue or pull request already exists

  
Reviewed
Invalid
 Invalid issue

  
Reviewed
Won't Fix
 This issue won't be fixed

  
Status
Abandoned
 Somebody has started to work on this but abandoned work

  
Status
Blocked
 Something is blocking this issue or pull request

  
Status
Need More Info
 Feedback is required to reproduce issue or to continue work

No labels
code-review
correctness
dos
performance
security
severity/high
severity/low
severity/medium
tech-debt
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Depends on
#78 index: generation-safe plugin activation, invalidation, promotion and rollback
h-dv/code-index
#80 gate: plugin conformance, hostile-runtime containment and production disclosure
h-dv/code-index
Reference
h-dv/code-index#42
No description provided.