h-dv/code-index

Fork 0

test: corpus mutation guard — deletion/rename/injection phantom tripwires at real-repo ambiguity #43

New issue

Closed

opened 2026-07-28 20:38:31 +02:00 by buildagent · 3 comments

buildagent commented

2026-07-28 20:38:31 +02:00

Member

P2 — self-oracling precision guard. Measured 0-defect baseline.

From _prdoc/records/brainstorm-2026-07-28-oss-corpus-test-system.md §2.2. Depends on #40.

Why this is worth having even though it found nothing

The 2026-07-28 spike ran the deletion mutation against 3 real repos and found 0 rebinds out of 1413 previously-resolved cross-file refs (ripgrep crates/core/flags/parse.rs 568 refs, guzzle src/Handler/CurlFactory.php 787, sinatra lib/sinatra/indifferent_hash.rb 58). The resolver's honesty held.

Its value is therefore regression protection at an ambiguity level the fixtures cannot reach, not discovery.

Concretely: today's precision_gate.rs decoys sit in pools of ~1 candidate because the fixtures are 4–14 files — the exact vacuity I033 had to add a structural anti-vacuity guard against. In ts-zod (8489 symbols) or ripgrep (4127) a decoy competes against hundreds of real same-name candidates, deep import graphs, and macro noise. Same oracle cost, far higher discriminating power.

Three self-oracling mutation families

Deletion — delete the definition file absorbing the most cross-file resolved refs; every ref that resolved to it must become unresolved. Silently rebinding to a same-named neighbour is a phantom — the exact shape of the I026 and I037 defects.
Rename — rename a definition; refs follow it or go unresolved, never bind to the old name's neighbours.
Injection — stage a uniquely-named symbol plus call sites at known coordinates into a real repo; it must resolve exactly there and nothing else may bind to it. Generalises write_phantom_decoys from precision_gate.rs to corpus scale.

All three need no external tooling — ground truth is known by construction.

Method notes from the spike

compare via ATTACH across the pre- and post-mutation DBs, joining ref sites on (path, start_line, start_col, name)
exclude sites inside the victim file itself; only cross-file refs are meaningful
the honest expected answer after deleting a definition is unresolved, not "some other same-named symbol"

Acceptance

all three mutation families over ≥3 languages incl. ≥2 dynamic ones
rebind count asserted == 0, with the offending site printed on failure
positive control: assert the mutation actually landed (#44) — the spike's near-miss was exactly this
baseline of 0 rebinds / 1413 refs recorded so a future regression is visible

**P2 — self-oracling precision guard. Measured 0-defect baseline.** From `_prdoc/records/brainstorm-2026-07-28-oss-corpus-test-system.md` §2.2. Depends on #40. ## Why this is worth having even though it found nothing The 2026-07-28 spike ran the deletion mutation against 3 real repos and found **0 rebinds out of 1413 previously-resolved cross-file refs** (ripgrep `crates/core/flags/parse.rs` 568 refs, guzzle `src/Handler/CurlFactory.php` 787, sinatra `lib/sinatra/indifferent_hash.rb` 58). The resolver's honesty held. Its value is therefore **regression protection at an ambiguity level the fixtures cannot reach**, not discovery. Concretely: today's `precision_gate.rs` decoys sit in pools of ~1 candidate because the fixtures are 4–14 files — the exact vacuity I033 had to add a structural anti-vacuity guard against. In ts-zod (8489 symbols) or ripgrep (4127) a decoy competes against hundreds of real same-name candidates, deep import graphs, and macro noise. **Same oracle cost, far higher discriminating power.** ## Three self-oracling mutation families 1. **Deletion** — delete the definition file absorbing the most cross-file resolved refs; every ref that resolved to it must become *unresolved*. Silently rebinding to a same-named neighbour is a phantom — the exact shape of the I026 and I037 defects. 2. **Rename** — rename a definition; refs follow it or go unresolved, never bind to the old name's neighbours. 3. **Injection** — stage a uniquely-named symbol plus call sites at known coordinates into a real repo; it must resolve exactly there and nothing else may bind to it. Generalises `write_phantom_decoys` from `precision_gate.rs` to corpus scale. All three need **no external tooling** — ground truth is known by construction. ## Method notes from the spike - compare via `ATTACH` across the pre- and post-mutation DBs, joining ref sites on `(path, start_line, start_col, name)` - exclude sites inside the victim file itself; only cross-file refs are meaningful - the honest expected answer after deleting a definition is *unresolved*, not "some other same-named symbol" ## Acceptance - [ ] all three mutation families over ≥3 languages incl. ≥2 dynamic ones - [ ] rebind count asserted == 0, with the offending site printed on failure - [ ] positive control: assert the mutation actually landed (#44) — the spike's near-miss was exactly this - [ ] baseline of 0 rebinds / 1413 refs recorded so a future regression is visible

buildagent referenced this issue

2026-07-28 20:39:29 +02:00

feat: gap miner — rank unresolved refs into a mission queue clamped by phantom==0 #47

buildagent added the

labels

2026-07-28 20:40:45 +02:00

buildagent commented

2026-07-28 21:36:19 +02:00

Author

Member

Landed — `crates/indexer/tests/corpus_mutation.rs`

Two self-oracling mutation families over all 7 tier-1 repos:

corpus_deletion_never_rebinds — deletes the file whose symbols absorb the most cross-file resolved refs, re-indexes, and requires every ref that resolved into it to become unresolved.

corpus_rename_never_binds_elsewhere — renames that file (keeping the extension) and requires each ref to follow it to the new path or decline. Binding to a third file is the phantom.

Result

mutation-delete: 0 rebind(s) across 7056 previously-resolved cross-file site(s)
mutation-rename: 0 stray binding(s) across 7056 site(s)

7056 sites across all 7 languages, up from the spike's 1413 across 3. Zero rebinds, zero strays. The resolver's honesty holds at real-repo ambiguity.

Two implementation notes worth keeping

Both mutations re-index cold, deliberately. The incremental path carries the separate stickiness defect found in #52; mixing them would confound a precision result with a coherence one.
The rename must preserve the file extension. Renaming to *.renamed makes the walker stop recognising the language, so the file leaves the index entirely — a deletion wearing a rename's clothes. This was caught by a positive control on the very first run of the sibling suite in #42.

Positive controls (#44)

14 per suite: the victim actually absorbs cross-file bindings (an empty site list makes the assertion vacuously true), and the mutation actually landed in the re-index.

Acceptance boxes 1–4 met. Baseline of 0/7056 recorded here so a regression is visible.

## Landed — `crates/indexer/tests/corpus_mutation.rs` Two self-oracling mutation families over all 7 tier-1 repos: **`corpus_deletion_never_rebinds`** — deletes the file whose symbols absorb the most cross-file resolved refs, re-indexes, and requires every ref that resolved into it to become unresolved. **`corpus_rename_never_binds_elsewhere`** — renames that file (keeping the extension) and requires each ref to follow it to the new path or decline. Binding to a *third* file is the phantom. ## Result ``` mutation-delete: 0 rebind(s) across 7056 previously-resolved cross-file site(s) mutation-rename: 0 stray binding(s) across 7056 site(s) ``` **7056 sites across all 7 languages**, up from the spike's 1413 across 3. Zero rebinds, zero strays. The resolver's honesty holds at real-repo ambiguity. ## Two implementation notes worth keeping - **Both mutations re-index cold, deliberately.** The incremental path carries the separate stickiness defect found in #52; mixing them would confound a precision result with a coherence one. - **The rename must preserve the file extension.** Renaming to `*.renamed` makes the walker stop recognising the language, so the file leaves the index entirely — a deletion wearing a rename's clothes. This was caught by a positive control on the very first run of the sibling suite in #42. ## Positive controls (#44) 14 per suite: the victim actually absorbs cross-file bindings (an empty site list makes the assertion vacuously true), and the mutation actually landed in the re-index. Acceptance boxes 1–4 met. Baseline of 0/7056 recorded here so a regression is visible.

buildagent referenced this issue

2026-07-29 21:26:30 +02:00

I039: OSS corpus test system + the four defects it found (v0.9.0, schema v25) #58

buildagent referenced this issue from a commit

2026-08-03 14:06:57 +02:00

test: OSS corpus test system — metamorphic, mutation, live-watcher replay (#39-#44)

buildagent referenced this issue from a commit

2026-08-03 14:06:57 +02:00

I039: OSS corpus test system + resolver fixes (v0.9.0, schema v25) (#58)

dhoyer referenced this issue from a commit

2026-08-07 08:21:17 +02:00

feat(daemon): log startup WAL checkpoint elapsed_ms (#43)

buildagent commented

2026-08-07 21:27:19 +02:00

Author

Member

Status check — NOT closing. Two of the three mutation families shipped; injection was never built.

Acceptance item 1 requires "all three mutation families". crates/indexer/tests/corpus_mutation.rs covers deletion and rename (corpus_deletion_never_rebinds, corpus_rename_never_binds_elsewhere, both green with the corpus armed, both registering positive controls per #44). The injection family — stage a uniquely-named symbol plus call sites at known coordinates, assert it resolves exactly there and nothing else binds to it — has no implementation in the tree.

That third family is the one that generalises write_phantom_decoys from precision_gate.rs to corpus scale, and it is arguably the highest-value of the three: this issue's own rationale is that fixture decoys sit in ~1-candidate pools while a corpus decoy competes against hundreds of real same-name candidates. Deletion and rename prove the resolver doesn't rebind; only injection proves it doesn't over-bind at ambiguity.

Recommend one of: build the injection family and then close, or scope this issue down to the two shipped families and re-file injection as its own item so the gap stays visible. Leaving it open as-is risks it reading like a finished suite.

**Status check — NOT closing. Two of the three mutation families shipped; `injection` was never built.** Acceptance item 1 requires *"all three mutation families"*. `crates/indexer/tests/corpus_mutation.rs` covers **deletion** and **rename** (`corpus_deletion_never_rebinds`, `corpus_rename_never_binds_elsewhere`, both green with the corpus armed, both registering positive controls per #44). The **injection** family — stage a uniquely-named symbol plus call sites at known coordinates, assert it resolves exactly there and nothing else binds to it — has no implementation in the tree. That third family is the one that generalises `write_phantom_decoys` from `precision_gate.rs` to corpus scale, and it is arguably the highest-value of the three: this issue's own rationale is that fixture decoys sit in ~1-candidate pools while a corpus decoy competes against hundreds of real same-name candidates. Deletion and rename prove the resolver doesn't *rebind*; only injection proves it doesn't *over-bind* at ambiguity. Recommend one of: build the injection family and then close, or scope this issue down to the two shipped families and re-file injection as its own item so the gap stays visible. Leaving it open as-is risks it reading like a finished suite.

dhoyer referenced this issue from a commit

2026-08-07 22:18:26 +02:00

test(corpus): injection mutation family, PR-gated corpus, fetch integrity

dhoyer referenced this issue from a commit

2026-08-07 22:18:26 +02:00

Merge: corpus injection family, PR-gated corpus, fetch integrity

dhoyer closed this issue

2026-08-07 22:18:26 +02:00

buildagent commented

2026-08-07 22:19:44 +02:00

Author

Member

Closing — the injection family landed in a6b97c0, so all four acceptance items are now met.

1. All three families, ≥3 languages incl. ≥2 dynamic — corpus_injection_binds_only_where_staged joins the two existing families. 7/7 tier-1 repos execute, all 7 languages, 4 dynamic. The floor is asserted in code, not just satisfied by luck:

langs.len() >= 3 && dynamic.len() >= 2

so the family cannot silently shrink to one language.

2. Rebind count asserted == 0, offending site printed — assert!(rebinds.is_empty(), "…PHANTOM rebind(s) after deleting {victim}…"). Demonstrated live by the M1 mutation, which printed real offending sites:

rust-ripgrep: 599 PRE-EXISTING ref(s) bound to the INJECTED definition file …
  crates/core/flags/config.rs:133:9 `assert_eq` -> assert_eq

3. Positive controls (#44) — 4 per repo, 28 total: the site is dense not a quiet corner; the injected names are unused pre-injection; the injection landed; the call sites landed at their exact coordinates.

4. Baseline recorded — the 0 rebinds / 1413 refs figure is in the module doc, and every run prints its live equivalent (mutation-delete: N rebind(s) across M previously-resolved cross-file site(s)), so a regression is visible in the log rather than only in a failure.

On what makes injection worth the effort — this issue argued it generalises write_phantom_decoys to corpus scale, and that is what it does. The probe is staged into each repo's densest directory (ripgrep 1315 syms / 370 names, ts-zod 1504 / 1185, guzzle 1326 / 1194), so it competes against hundreds of real same-name candidates. A unique name cannot be reached by name, so the teeth are in the paths that bind without consulting the name — same-dir locality, tier-1Q anchors, tier-1R receivers. The sharp assertion is inertness: 230,508 foreign bindings proved bit-identical before and after across the 7 repos.

The mutation proof worth reading — neutering positive-control 4 makes the whole suite pass green having graded nothing (the positive loop iterates over zero targets). That is exactly the near-miss this issue and #44 were filed about, demonstrated rather than asserted, and it is why the control exists.

Honest gaps, recorded in the test's own doc comment rather than here only: the distant import-less call site is UNRESOLVED in all 7 repos and is graded by the negative only (requiring it to resolve would assert recall the resolver does not claim); python-flask and js-express inject into test trees because those genuinely are their densest directories; and this is cold-index only, matching the other two families.

Closing — the injection family landed in `a6b97c0`, so all four acceptance items are now met. **1. All three families, ≥3 languages incl. ≥2 dynamic** — `corpus_injection_binds_only_where_staged` joins the two existing families. **7/7 tier-1 repos execute, all 7 languages, 4 dynamic.** The floor is asserted in code, not just satisfied by luck: ```rust langs.len() >= 3 && dynamic.len() >= 2 ``` so the family cannot silently shrink to one language. **2. Rebind count asserted == 0, offending site printed** — `assert!(rebinds.is_empty(), "…PHANTOM rebind(s) after deleting {victim}…")`. Demonstrated live by the M1 mutation, which printed real offending sites: ``` rust-ripgrep: 599 PRE-EXISTING ref(s) bound to the INJECTED definition file … crates/core/flags/config.rs:133:9 `assert_eq` -> assert_eq ``` **3. Positive controls (#44)** — 4 per repo, 28 total: the site is dense not a quiet corner; the injected names are unused pre-injection; the injection landed; the call sites landed at their exact coordinates. **4. Baseline recorded** — the `0 rebinds / 1413 refs` figure is in the module doc, and every run prints its live equivalent (`mutation-delete: N rebind(s) across M previously-resolved cross-file site(s)`), so a regression is visible in the log rather than only in a failure. **On what makes injection worth the effort** — this issue argued it generalises `write_phantom_decoys` to corpus scale, and that is what it does. The probe is staged into each repo's *densest* directory (ripgrep 1315 syms / 370 names, ts-zod 1504 / 1185, guzzle 1326 / 1194), so it competes against hundreds of real same-name candidates. A unique name cannot be reached *by name*, so the teeth are in the paths that bind without consulting the name — same-dir locality, tier-1Q anchors, tier-1R receivers. The sharp assertion is inertness: **230,508 foreign bindings proved bit-identical before and after** across the 7 repos. **The mutation proof worth reading** — neutering positive-control 4 makes the whole suite pass green *having graded nothing* (the positive loop iterates over zero targets). That is exactly the near-miss this issue and #44 were filed about, demonstrated rather than asserted, and it is why the control exists. Honest gaps, recorded in the test's own doc comment rather than here only: the distant import-less call site is UNRESOLVED in all 7 repos and is graded by the negative only (requiring it to resolve would assert recall the resolver does not claim); python-flask and js-express inject into test trees because those genuinely are their densest directories; and this is cold-index only, matching the other two families.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

h-dv/code-index#43

No description provided.

Rows
Columns

test: corpus mutation guard — deletion/rename/injection phantom tripwires at real-repo ambiguity #43

Why this is worth having even though it found nothing

Three self-oracling mutation families

Method notes from the spike

Acceptance

Landed — crates/indexer/tests/corpus_mutation.rs

Result

Two implementation notes worth keeping

Positive controls (#44)

Landed — `crates/indexer/tests/corpus_mutation.rs`