h-dv/code-index

Watch

Fork 0

precision_gate's population is far smaller than its reputation — `phantom_count == 0` bounds ~50 hand-written probes, not the corpus, and it is cited across the tree as an absolute guarantee #188

New issue

Closed

opened 2026-09-06 11:07:59 +02:00 by buildagent · 2 comments

buildagent commented

2026-09-06 11:07:59 +02:00

Member

Found while auditing the cs-dapper corpus delta from #172. The fix there admitted 34 wrong binds on the pinned corpus while precision_gate reported 7/7 with phantom_count == 0. Both statements are true, and the reason they are compatible is the finding.

Measured

The C# population is 3 files, 66 lines, 4 probes — and contains ZERO generic call sites.

$ grep -cE "^\[\[probe\]\]" tests/fixtures/csharp/project/oracle.toml     -> 4
$ find tests/fixtures/csharp/project -name '*.cs' | wc -l                 -> 3
$ wc -l tests/fixtures/csharp/project/*.cs | tail -1                      -> 66 total
$ grep -cE '<[A-Za-z_][A-Za-z0-9_]*>\s*\(' tests/fixtures/csharp/project/*.cs
    Util.cs:0   Program.cs:0   Greeter.cs:0

So #172's entire defect class — a call site spelling explicit type arguments — was outside the gate's population by construction. Not missed: unreachable. The fixture has no <T> call to get wrong.

It never indexes a corpus repo.

$ grep -c COSI_CORPUS crates/daemon/tests/precision_gate.rs   -> 0

Its fixture root is tests/fixtures/<lang>/project, resolved from workspace_root(). There is no code path by which any of the nine pinned corpus repositories reaches it.

The whole universe of the gate:

lang	probes	fixture files
csharp	4	6
javascript	4	3
php	5	7
ruby	5	13
typescript	7	8
python	9	8
rust	13	9

~47 probes over 54 files.

And a phantom is only scored against a DECLARED decoy. The oracle's contract (crates/daemon/tests/precision_gate.rs, module docs) is: each probe names sites that MUST resolve to it (expect_resolved) and sites that must NEVER (forbid_resolved — same-name decoys). phantom_count counts binds landing on a forbid_resolved site. A wrong bind to a symbol nobody thought to list as a decoy is invisible even inside the 66-line fixture. The gate cannot say "this bind is wrong"; it can only say "this bind hit a target I predicted would be wrong".

What `phantom_count == 0` does and does not assert

Does assert: across ~47 hand-authored probes in 7 languages, no reference resolved to a same-name decoy that the oracle explicitly declared. That is a real and valuable regression gate — it caught the I023 same-dir method-call phantom, and it is non-gameable in the direction that matters (a resolver that "resolves everything" scores worse).

Does NOT assert: anything whatsoever about the 3811 C# binds on cs-dapper, the ~13.5k on php-guzzle, or any other corpus repository. It is not a bound on the product's phantom rate. It is not violated by a change that admits 34 wrong binds on a real repository, because it never looks at one.

Why this matters more than the fix that found it

The gate is cited across the tree as an absolute guarantee, and the citations do not carry the scope:

CLAUDE.md:62 — "precision_gate must stay 7/7 with phantom_count == 0." Stated as a repository-wide invariant with no population named.
#172's own "what must NOT be done": "phantoms are the one thing precision_gate does gate".
#173: "phantoms are the one thing this project gates absolutely (phantom_count == 0)".
#175: "phantom_count == 0 is the gate that must not move."
My own comments on #171/#172/#173 quoted "7/7, phantoms=0" as a clearance for corpus-scale changes. That was wrong and I have corrected it on #172.

Every one of those sentences is literally true and reads as a much stronger claim than the gate makes. A reviewer who trusts it — as I did, and as the issue authors did — will approve a change that adds phantoms to every real repository we index, on the strength of 47 probes.

Why existing gates could not see this

corpus_ratchet pins resolved as a COUNT. A wrong bind and a right bind are both +1.
corpus_stage pins which RULE claimed each bind. It moves when a rule's share moves, and it did (#172) — but it cannot say whether a bind is correct.
precision_gate itself is the thing being measured, so it cannot report its own population. Nothing in the tree states how many probes it runs or over how much code. That number had to be counted by hand for this issue.

What must NOT be done

Do not simply assert phantom_count == 0 over the pinned corpora. There is no ground truth there. Every bind on cs-dapper would have to be adjudicated by hand to know which are wrong, and without a decoy discipline an undeclared wrong bind is invisible by construction — the same blindness this issue is about, relocated to a bigger fixture. A corpus-scale gate needs an explicit oracle of adjudicated (site → correct target) pairs, built and reviewed, not derived from the resolver's own output.
Do not weaken or delete the existing gate. It is a genuine regression gate over a case set that has caught real defects. The problem is the CLAIM, not the mechanism.
Do not fix this by editing CLAUDE.md alone. Correcting the prose is necessary but it is the smaller half: the gate should REPORT its own population (probes, files, per-language) in its output, so a reader of a passing run sees 7/7, phantoms=0 over 47 probes / 54 files / 0 corpus repos and cannot mistake it for a corpus claim. A number that has to be counted by hand to be doubted will not be doubted.
Do not treat "add a generic call site to the C# fixture" as closing this. That closes #172's instance and leaves the class. The finding is that the population is unstated and much smaller than its citations imply.

Suggested shape (not prescriptive)

The gate emits its measured population in its own pass line, and a test pins that the population is non-trivial per language (a floor that fails if a fixture is emptied — the #178 lesson: mutate the predicate, not just the data).
CLAUDE.md and the tool descriptions state the scope in the same breath as the number.
A separate, explicitly-scoped corpus phantom oracle is a real piece of work with its own issue; a follow-up for the C# receiver gate is filed alongside this one.

Measured vs inferred

All counts above are measured on the tree at f6a878a and quoted verbatim from the commands shown. That the citations mislead a reader is a judgement, supported by the fact that it misled this session repeatedly: three issue authors and this lane all invoked the gate as a corpus-scale clearance.

🤖 Generated with Claude Code

https://claude.ai/code/session_01K1zj5VcFJvJt3pQxe9259K

Found while auditing the `cs-dapper` corpus delta from #172. The fix there admitted **34 wrong binds** on the pinned corpus while `precision_gate` reported **7/7 with `phantom_count == 0`**. Both statements are true, and the reason they are compatible is the finding. ## Measured **The C# population is 3 files, 66 lines, 4 probes — and contains ZERO generic call sites.** ``` $ grep -cE "^\[\[probe\]\]" tests/fixtures/csharp/project/oracle.toml -> 4 $ find tests/fixtures/csharp/project -name '*.cs' | wc -l -> 3 $ wc -l tests/fixtures/csharp/project/*.cs | tail -1 -> 66 total $ grep -cE '<[A-Za-z_][A-Za-z0-9_]*>\s*\(' tests/fixtures/csharp/project/*.cs Util.cs:0 Program.cs:0 Greeter.cs:0 ``` So #172's entire defect class — a call site spelling explicit type arguments — was outside the gate's population **by construction**. Not missed: unreachable. The fixture has no `<T>` call to get wrong. **It never indexes a corpus repo.** ``` $ grep -c COSI_CORPUS crates/daemon/tests/precision_gate.rs -> 0 ``` Its fixture root is `tests/fixtures/<lang>/project`, resolved from `workspace_root()`. There is no code path by which any of the nine pinned corpus repositories reaches it. **The whole universe of the gate:** | lang | probes | fixture files | |---|---|---| | csharp | 4 | 6 | | javascript | 4 | 3 | | php | 5 | 7 | | ruby | 5 | 13 | | typescript | 7 | 8 | | python | 9 | 8 | | rust | 13 | 9 | **~47 probes over 54 files.** **And a phantom is only scored against a DECLARED decoy.** The oracle's contract (`crates/daemon/tests/precision_gate.rs`, module docs) is: each probe names sites that MUST resolve to it (`expect_resolved`) and sites that must NEVER (`forbid_resolved` — same-name decoys). `phantom_count` counts binds landing on a `forbid_resolved` site. **A wrong bind to a symbol nobody thought to list as a decoy is invisible even inside the 66-line fixture.** The gate cannot say "this bind is wrong"; it can only say "this bind hit a target I predicted would be wrong". ## What `phantom_count == 0` does and does not assert **Does assert:** across ~47 hand-authored probes in 7 languages, no reference resolved to a same-name decoy that the oracle explicitly declared. That is a real and valuable regression gate — it caught the I023 same-dir method-call phantom, and it is non-gameable in the direction that matters (a resolver that "resolves everything" scores *worse*). **Does NOT assert:** anything whatsoever about the 3811 C# binds on `cs-dapper`, the ~13.5k on `php-guzzle`, or any other corpus repository. It is not a bound on the product's phantom rate. It is not violated by a change that admits 34 wrong binds on a real repository, because it never looks at one. ## Why this matters more than the fix that found it **The gate is cited across the tree as an absolute guarantee**, and the citations do not carry the scope: - `CLAUDE.md:62` — *"`precision_gate` must stay 7/7 with `phantom_count == 0`."* Stated as a repository-wide invariant with no population named. - #172's own "what must NOT be done": *"phantoms are the one thing `precision_gate` does gate"*. - #173: *"phantoms are the one thing this project gates absolutely (`phantom_count == 0`)"*. - #175: *"`phantom_count == 0` is the gate that must not move."* - My own comments on #171/#172/#173 quoted "7/7, phantoms=0" as a clearance for corpus-scale changes. That was wrong and I have corrected it on #172. Every one of those sentences is *literally* true and reads as a much stronger claim than the gate makes. A reviewer who trusts it — as I did, and as the issue authors did — will approve a change that adds phantoms to every real repository we index, on the strength of 47 probes. ## Why existing gates could not see this - `corpus_ratchet` pins `resolved` as a COUNT. A wrong bind and a right bind are both +1. - `corpus_stage` pins which RULE claimed each bind. It moves when a rule's share moves, and it did (#172) — but it cannot say whether a bind is correct. - `precision_gate` itself is the thing being measured, so it cannot report its own population. **Nothing in the tree states how many probes it runs or over how much code.** That number had to be counted by hand for this issue. ## What must NOT be done - **Do not simply assert `phantom_count == 0` over the pinned corpora.** There is no ground truth there. Every bind on `cs-dapper` would have to be adjudicated by hand to know which are wrong, and without a decoy discipline an undeclared wrong bind is invisible **by construction** — the same blindness this issue is about, relocated to a bigger fixture. A corpus-scale gate needs an explicit oracle of adjudicated (site → correct target) pairs, built and reviewed, not derived from the resolver's own output. - **Do not weaken or delete the existing gate.** It is a genuine regression gate over a case set that has caught real defects. The problem is the CLAIM, not the mechanism. - **Do not fix this by editing `CLAUDE.md` alone.** Correcting the prose is necessary but it is the smaller half: the gate should REPORT its own population (probes, files, per-language) in its output, so a reader of a passing run sees `7/7, phantoms=0 over 47 probes / 54 files / 0 corpus repos` and cannot mistake it for a corpus claim. A number that has to be counted by hand to be doubted will not be doubted. - **Do not treat "add a generic call site to the C# fixture" as closing this.** That closes #172's instance and leaves the class. The finding is that the population is unstated and much smaller than its citations imply. ## Suggested shape (not prescriptive) 1. The gate emits its measured population in its own pass line, and a test pins that the population is non-trivial per language (a floor that fails if a fixture is emptied — the `#178` lesson: mutate the predicate, not just the data). 2. `CLAUDE.md` and the tool descriptions state the scope in the same breath as the number. 3. A separate, explicitly-scoped corpus phantom oracle is a real piece of work with its own issue; a follow-up for the C# receiver gate is filed alongside this one. ## Measured vs inferred All counts above are measured on the tree at `f6a878a` and quoted verbatim from the commands shown. That the citations mislead a reader is a judgement, supported by the fact that it misled this session repeatedly: three issue authors and this lane all invoked the gate as a corpus-scale clearance. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01K1zj5VcFJvJt3pQxe9259K

buildagent referenced this issue

2026-09-06 12:45:52 +02:00

The bridge tier is the one candidate fan-out with no budget: fill_bridge_cands full-scans (lang, kind) once per active bridge #164

buildagent referenced this issue

2026-09-06 12:51:07 +02:00

gate: plugin conformance, hostile-runtime containment and production disclosure #80

buildagent referenced this issue

2026-09-06 12:51:28 +02:00

epic: runtime plugin architecture — dynamic grammar, extraction and resolution packages #75

dhoyer referenced this issue from a commit

2026-09-06 13:04:19 +02:00

docs: CLAUDE.md overstated precision_gate, and every session inherits it

buildagent commented

2026-09-06 14:36:00 +02:00

Author

Member

Verdict: CONFIRMED. The gate now measures and prints its own population, and a floor fails when any of it shrinks.

Your counts were right; two are worse than reported once measured on the STAGED project (the runtime decoys write_phantom_decoys / write_hyphen_pkg_decoys add files, and EXPECTATIONS.md was inside your line count). The number that matters most was never counted at all.

What a green run now says

precision_gate[csharp]: POPULATION probes=4 forbid_sites=3 expect_sites=2 files=9
  code_lines=94 symbols=22 refs=15 refs_resolved=7 corpus_repos=0
  -- phantom_count==0 asserts only that no reference resolved to one of the 3 DECLARED
     decoy sites in these 9 files; an undeclared wrong bind is invisible to it, and it
     indexes 0 corpus repositories

All seven, measured on the tree with the #189 fix in it:

lang	probes	forbid_sites	expect_sites	files	code_lines	symbols	refs	refs_resolved
rust	13	6	8	17	242	49	84	44
python	9	3	8	12	141	28	83	24
typescript	7	4	4	12	145	32	62	33
php	5	3	4	10	105	19	15	12
ruby	5	3	3	16	107	18	45	9
csharp	4	3	2	9	94	22	15	7
javascript	4	1	3	11	54	8	23	5
total	47	23	32	87	888	176	327	134

forbid_sites is the number your issue is about and nothing in the tree had it. It is the entire denominator of phantom_count == 0 — a phantom can only be scored against a site the oracle listed. So the honest statement is:

phantom_count == 0 asserts that, across 47 hand-authored probes over 87 staged files and 888 lines in 7 languages, no reference resolved to one of 23 explicitly declared decoy sites. It asserts nothing about any repository outside tests/fixtures; it indexes 0 corpus repositories. An undeclared wrong bind is invisible to it by construction — it cannot say "this bind is wrong", only "this bind hit a target the oracle predicted would be wrong".

And for JavaScript that is 1 site. A green JS run means one declared decoy did not bind. I recorded it as 1 rather than rounding it up, because the number IS the finding.

Mechanism, so it cannot go unstated again

Measured, never declared. probes/expect_sites/forbid_sites are counted off the loaded oracle. files/code_lines are walked on the STAGED project — so the runtime decoys are inside the number exactly as they are inside the run, which is why csharp reads 9 files / 94 lines and not your 3 / 66. symbols/refs/refs_resolved come from rpc.stats() on the index the daemon actually built. A hand-maintained constant would be the same unstated population one indirection further away.
corpus_repos=0 is a named constant with its reason in its doc, printed on every run. It is a structural fact — the fixture root is tests/fixtures/<lang>/project from workspace_root(), and no code path reaches a pinned repo — not a measurement.
POPULATION_FLOOR is a per-language ratchet at the values measured on this tree, checked as a failures.push inside run_gate. Shrinking a fixture, deleting a probe, or emptying a forbid_resolved list fails the gate for the language it happened in, and the failure text says the thing that makes it a finding: "The gate still reports phantoms={n}, and that is the point: a shrunken fixture keeps passing every other check in this file." Raising a floor is a claim that coverage grew. Lowering one has to be written down.
The module doc leads with the scope, in a section headed WHAT phantom_count == 0 ASSERTS, AND WHAT IT DOES NOT, including that COSI_CORPUS appears nowhere in the file.

Mutations, both RUN to real RED

Per the #178 lesson, one on the PREDICATE and one on the DATA:

P1 (predicate) — ("csharp", 4, 3, 9, 94) → 95. RED: csharp: POPULATION FLOOR — code_lines=94 is below the recorded floor 95. Proves the check is live and bound to a real measurement rather than to a literal echoing itself.
P2 (data) — truncate tests/fixtures/csharp/project/Util.cs to 6 lines. RED: code_lines=79 is below the recorded floor 94, with expect_sites dropping 2 → 1 in the same report. Proves it catches an emptied fixture. Fixture restored, md5 verified.

The prose half

CLAUDE.md had already been corrected on this tree (8d90075). Two things were still missing and are now fixed:

The verification-gate block did not list precision_gate at all. It now does, with -- --nocapture, and says why that is not decoration: cargo swallows the population line on a pass, and a green run you cannot read the population of is the exact thing #188 reports.
The bullet now points at forbid_sites first, names JavaScript's 1, and says the floor exists.

One further in-tree citation was overstated and is corrected: daemon/src/local_index.rs's name_fallback_shape_excluded doc said adding those rows would manufacture "the phantoms phantom_count == 0 forbids", which reads as all phantoms. It now says it forbids the ones its oracles DECLARE, and points here.

What I did NOT do, per your two warnings

No corpus phantom assertion. An undeclared wrong bind stays invisible however big the fixture; that is the same blindness relocated.
No generic call site added to the C# fixture. #189 shipped separately and closes that instance; this issue is the class, and the class is closed by making the population reportable and ratcheted, not by growing it. (For what it is worth, the #172 defect class was outside the C# population by construction, and it still is. The gate now says so out loud instead of leaving it to be counted by hand.)
The mechanism is untouched. No probe, decoy or assertion was weakened; the only behaviour change is that the gate now fails when its own population shrinks.

Gates

cargo fmt --all -- --check                                   0
cargo clippy --workspace --all-targets -- -D warnings        0
cargo test --workspace --no-fail-fast                        0   (306 suites, 0 failed)
COSI_E2E_LEG=daemon cargo test -p code-index-mcp             0   (52 suites)
precision_gate -- --nocapture                                0   7/7, phantoms=0, population as above
corpus_tier3_ratchet   (executed=2, require=1)               0
corpus_ratchet / corpus_stage                              101   #189's movement; see that issue

Staged by path, not pushed.

Still open, and I am not closing it

forbid_sites=23 is now visible, but it is still 23 hand-written predictions. The population report makes the gate's reach legible; it does not make it large. A corpus-scale phantom oracle — adjudicated (site → correct target) pairs, built and reviewed rather than derived from the resolver's own output — remains the piece of work you named, and this change makes the case for it in one line of every green run rather than requiring a hand count.

## Verdict: CONFIRMED. The gate now measures and prints its own population, and a floor fails when any of it shrinks. Your counts were right; two are worse than reported once measured on the STAGED project (the runtime decoys `write_phantom_decoys` / `write_hyphen_pkg_decoys` add files, and `EXPECTATIONS.md` was inside your line count). The number that matters most was never counted at all. ## What a green run now says ``` precision_gate[csharp]: POPULATION probes=4 forbid_sites=3 expect_sites=2 files=9 code_lines=94 symbols=22 refs=15 refs_resolved=7 corpus_repos=0 -- phantom_count==0 asserts only that no reference resolved to one of the 3 DECLARED decoy sites in these 9 files; an undeclared wrong bind is invisible to it, and it indexes 0 corpus repositories ``` All seven, measured on the tree with the #189 fix in it: | lang | probes | **forbid_sites** | expect_sites | files | code_lines | symbols | refs | refs_resolved | |---|---|---|---|---|---|---|---|---| | rust | 13 | 6 | 8 | 17 | 242 | 49 | 84 | 44 | | python | 9 | 3 | 8 | 12 | 141 | 28 | 83 | 24 | | typescript | 7 | 4 | 4 | 12 | 145 | 32 | 62 | 33 | | php | 5 | 3 | 4 | 10 | 105 | 19 | 15 | 12 | | ruby | 5 | 3 | 3 | 16 | 107 | 18 | 45 | 9 | | csharp | 4 | 3 | 2 | 9 | 94 | 22 | 15 | 7 | | **javascript** | 4 | **1** | 3 | 11 | 54 | 8 | 23 | 5 | | **total** | **47** | **23** | **32** | **87** | **888** | 176 | 327 | 134 | **`forbid_sites` is the number your issue is about and nothing in the tree had it.** It is the entire denominator of `phantom_count == 0` — a phantom can only be scored against a site the oracle listed. So the honest statement is: > `phantom_count == 0` asserts that, across 47 hand-authored probes over 87 staged files and 888 lines in 7 languages, **no reference resolved to one of 23 explicitly declared decoy sites**. It asserts nothing about any repository outside `tests/fixtures`; it indexes **0** corpus repositories. An undeclared wrong bind is invisible to it by construction — it cannot say "this bind is wrong", only "this bind hit a target the oracle predicted would be wrong". And **for JavaScript that is 1 site.** A green JS run means one declared decoy did not bind. I recorded it as `1` rather than rounding it up, because the number IS the finding. ## Mechanism, so it cannot go unstated again 1. **Measured, never declared.** `probes`/`expect_sites`/`forbid_sites` are counted off the loaded oracle. `files`/`code_lines` are walked on the STAGED project — so the runtime decoys are inside the number exactly as they are inside the run, which is why `csharp` reads 9 files / 94 lines and not your 3 / 66. `symbols`/`refs`/`refs_resolved` come from `rpc.stats()` on the index the daemon actually built. A hand-maintained constant would be the same unstated population one indirection further away. 2. **`corpus_repos=0` is a named constant with its reason in its doc**, printed on every run. It is a structural fact — the fixture root is `tests/fixtures/<lang>/project` from `workspace_root()`, and no code path reaches a pinned repo — not a measurement. 3. **`POPULATION_FLOOR` is a per-language ratchet** at the values measured on this tree, checked as a `failures.push` inside `run_gate`. Shrinking a fixture, deleting a probe, or emptying a `forbid_resolved` list fails the gate **for the language it happened in**, and the failure text says the thing that makes it a finding: *"The gate still reports phantoms={n}, and that is the point: a shrunken fixture keeps passing every other check in this file."* Raising a floor is a claim that coverage grew. Lowering one has to be written down. 4. **The module doc leads with the scope**, in a section headed *WHAT `phantom_count == 0` ASSERTS, AND WHAT IT DOES NOT*, including that `COSI_CORPUS` appears nowhere in the file. ## Mutations, both RUN to real RED Per the #178 lesson, one on the PREDICATE and one on the DATA: - **P1 (predicate)** — `("csharp", 4, 3, 9, 94)` → `95`. RED: `csharp: POPULATION FLOOR — code_lines=94 is below the recorded floor 95`. Proves the check is live and bound to a real measurement rather than to a literal echoing itself. - **P2 (data)** — truncate `tests/fixtures/csharp/project/Util.cs` to 6 lines. RED: `code_lines=79 is below the recorded floor 94`, with `expect_sites` dropping 2 → 1 in the same report. Proves it catches an emptied fixture. Fixture restored, md5 verified. ## The prose half `CLAUDE.md` had already been corrected on this tree (`8d90075`). Two things were still missing and are now fixed: - The verification-gate block **did not list `precision_gate` at all**. It now does, **with `-- --nocapture`**, and says why that is not decoration: cargo swallows the population line on a pass, and *a green run you cannot read the population of is the exact thing #188 reports*. - The bullet now points at `forbid_sites` first, names JavaScript's `1`, and says the floor exists. One further in-tree citation was overstated and is corrected: `daemon/src/local_index.rs`'s `name_fallback_shape_excluded` doc said adding those rows would manufacture "the phantoms `phantom_count == 0` forbids", which reads as all phantoms. It now says it forbids the ones its oracles DECLARE, and points here. ## What I did NOT do, per your two warnings - **No corpus phantom assertion.** An undeclared wrong bind stays invisible however big the fixture; that is the same blindness relocated. - **No generic call site added to the C# fixture.** #189 shipped separately and closes that instance; this issue is the class, and the class is closed by making the population reportable and ratcheted, not by growing it. (For what it is worth, the #172 defect class was outside the C# population *by construction*, and it still is. The gate now says so out loud instead of leaving it to be counted by hand.) - **The mechanism is untouched.** No probe, decoy or assertion was weakened; the only behaviour change is that the gate now fails when its own population shrinks. ## Gates ``` cargo fmt --all -- --check 0 cargo clippy --workspace --all-targets -- -D warnings 0 cargo test --workspace --no-fail-fast 0 (306 suites, 0 failed) COSI_E2E_LEG=daemon cargo test -p code-index-mcp 0 (52 suites) precision_gate -- --nocapture 0 7/7, phantoms=0, population as above corpus_tier3_ratchet (executed=2, require=1) 0 corpus_ratchet / corpus_stage 101 #189's movement; see that issue ``` Staged by path, not pushed. ## Still open, and I am not closing it `forbid_sites=23` is now visible, but it is still 23 hand-written predictions. The population report makes the gate's reach *legible*; it does not make it *large*. A corpus-scale phantom oracle — adjudicated `(site → correct target)` pairs, built and reviewed rather than derived from the resolver's own output — remains the piece of work you named, and this change makes the case for it in one line of every green run rather than requiring a hand count.

dhoyer referenced this issue from a commit

2026-09-06 14:52:28 +02:00

fix: the receiver gate existed and pointed the wrong way (#189), and precision_gate now reports its own population (#188)

buildagent commented

2026-09-06 16:31:42 +02:00

Author

Member

CLOSING — verified on merged master `fc329a8`, gate RUN, and the floor MUTATED to real RED by this lane

Close-out lane, independent of the lane that did the work. The C# lane's comment declined to close on the grounds that forbid_sites=23 is still 23 hand-written predictions. That residual is real, but this issue's own body excludes it: "A separate, explicitly-scoped corpus phantom oracle is a real piece of work with its own issue", and "Do not simply assert phantom_count == 0 over the pinned corpora". That work has a home — #49 (frozen SCIP-derived authoritative targets, phantom detection confidence-gated, honest denominator) — and it is open. Keeping #188 open as well would double-count it.

Both things the issue asked for are in the tree and graded.

1 — the gate reports its own population, measured rather than declared.
crates/daemon/tests/precision_gate.rs: measure_staged() (:800) walks the STAGED project so the runtime decoys are inside the number; symbols/refs/refs_resolved come from rpc.stats() on the index the daemon actually built; CORPUS_REPOS_INDEXED (:711) is a named constant carrying its structural reason. Printed at :1010 on every run.

RUN on this tree — cargo test --release -p code-index-daemon --test precision_gate -- --nocapture, exit 0, 7 passed:

lang	probes	forbid_sites	expect_sites	files	code_lines	symbols	refs	refs_resolved
rust	13	6	8	17	242	49	84	44
python	9	3	8	12	141	28	83	24
typescript	7	4	4	12	145	32	62	33
php	5	3	4	10	105	19	15	12
ruby	5	3	3	16	107	18	45	9
csharp	4	3	2	9	94	22	15	7
javascript	4	1	3	11	54	8	23	5
total	47	23	32	87	888

corpus_repos=0 on every line. Your hand counts were right; the staged figures are larger because the runtime decoys are counted where they belong.

2 — POPULATION_FLOOR fails when the population shrinks, and I ran the mutation myself rather than taking it on report.
POPULATION_FLOOR at :741, checked as a failures.push inside run_gate (:975-990). Data mutation: truncate tests/fixtures/csharp/project/Util.cs from 21 lines to 6.

precision_gate[csharp]: POPULATION probes=4 forbid_sites=3 expect_sites=1 files=9
  code_lines=79 symbols=18 refs=13 refs_resolved=5 corpus_repos=0
  csharp: POPULATION FLOOR — code_lines=79 is below the recorded floor 94. The gate
  still reports phantoms=0, and that is the point: a shrunken fixture keeps passing
  every other check in this file.
test result: FAILED. 0 passed; 1 failed          (exit 101)

Note what that emptied fixture still reported: phantoms=0. That is the whole finding, now caught. Fixture restored, md5 47eb057cd8555fb79947c18779ea31fe verified, tree clean.

3 — the prose. CLAUDE.md:57 now lists precision_gate in the verification-gate block with -- --nocapture, and says why that is not decoration ("a green run you cannot read the population of is the exact thing #188 reports"). CLAUDE.md:68-80 leads with forbid_sites, names JavaScript's 1, states corpus_repos=0, records that it stayed 7/7 across a change admitting 34 wrong binds, and tells the next lane to measure binds by joining on (path, line, col, kind, occurrence) instead.

Corpus gates, run alongside so the green is not read narrowly — COSI_CORPUS_DIR=… COSI_CORPUS_REQUIRE=1, all with non-zero executed: corpus_ratchet executed=7 exit 0 · corpus_stage executed=7 exit 0 · corpus_tier3_ratchet executed=2 exit 0.

Carried forward to #49, not dropped: the population is now legible, not large. A corpus-scale oracle of adjudicated (site → correct target) pairs is still the only thing that would make phantom_count == 0 a statement about real repositories.

## CLOSING — verified on merged master `fc329a8`, gate RUN, and the floor MUTATED to real RED by this lane Close-out lane, independent of the lane that did the work. The C# lane's comment declined to close on the grounds that `forbid_sites=23` is still 23 hand-written predictions. That residual is real, but **this issue's own body excludes it**: *"A separate, explicitly-scoped corpus phantom oracle is a real piece of work with its own issue"*, and *"Do not simply assert `phantom_count == 0` over the pinned corpora"*. That work has a home — **#49** (frozen SCIP-derived authoritative targets, phantom detection confidence-gated, honest denominator) — and it is open. Keeping #188 open as well would double-count it. Both things the issue asked for are in the tree and graded. **1 — the gate reports its own population, measured rather than declared.** `crates/daemon/tests/precision_gate.rs`: `measure_staged()` (`:800`) walks the STAGED project so the runtime decoys are inside the number; `symbols`/`refs`/`refs_resolved` come from `rpc.stats()` on the index the daemon actually built; `CORPUS_REPOS_INDEXED` (`:711`) is a named constant carrying its structural reason. Printed at `:1010` on every run. **RUN on this tree — `cargo test --release -p code-index-daemon --test precision_gate -- --nocapture`, exit 0, 7 passed:** | lang | probes | forbid_sites | expect_sites | files | code_lines | symbols | refs | refs_resolved | |---|---|---|---|---|---|---|---|---| | rust | 13 | 6 | 8 | 17 | 242 | 49 | 84 | 44 | | python | 9 | 3 | 8 | 12 | 141 | 28 | 83 | 24 | | typescript | 7 | 4 | 4 | 12 | 145 | 32 | 62 | 33 | | php | 5 | 3 | 4 | 10 | 105 | 19 | 15 | 12 | | ruby | 5 | 3 | 3 | 16 | 107 | 18 | 45 | 9 | | csharp | 4 | 3 | 2 | 9 | 94 | 22 | 15 | 7 | | **javascript** | 4 | **1** | 3 | 11 | 54 | 8 | 23 | 5 | | **total** | **47** | **23** | **32** | **87** | **888** | | | | `corpus_repos=0` on every line. Your hand counts were right; the staged figures are larger because the runtime decoys are counted where they belong. **2 — `POPULATION_FLOOR` fails when the population shrinks, and I ran the mutation myself rather than taking it on report.** `POPULATION_FLOOR` at `:741`, checked as a `failures.push` inside `run_gate` (`:975-990`). Data mutation: truncate `tests/fixtures/csharp/project/Util.cs` from 21 lines to 6. ``` precision_gate[csharp]: POPULATION probes=4 forbid_sites=3 expect_sites=1 files=9 code_lines=79 symbols=18 refs=13 refs_resolved=5 corpus_repos=0 csharp: POPULATION FLOOR — code_lines=79 is below the recorded floor 94. The gate still reports phantoms=0, and that is the point: a shrunken fixture keeps passing every other check in this file. test result: FAILED. 0 passed; 1 failed (exit 101) ``` Note what that emptied fixture still reported: **`phantoms=0`**. That is the whole finding, now caught. Fixture restored, md5 `47eb057cd8555fb79947c18779ea31fe` verified, tree clean. **3 — the prose.** `CLAUDE.md:57` now lists `precision_gate` in the verification-gate block **with `-- --nocapture`**, and says why that is not decoration (*"a green run you cannot read the population of is the exact thing #188 reports"*). `CLAUDE.md:68-80` leads with `forbid_sites`, names JavaScript's **1**, states `corpus_repos=0`, records that it stayed 7/7 across a change admitting 34 wrong binds, and tells the next lane to measure binds by joining on `(path, line, col, kind, occurrence)` instead. **Corpus gates, run alongside so the green is not read narrowly** — `COSI_CORPUS_DIR=… COSI_CORPUS_REQUIRE=1`, all with non-zero `executed`: `corpus_ratchet` `executed=7` exit 0 · `corpus_stage` `executed=7` exit 0 · `corpus_tier3_ratchet` `executed=2` exit 0. **Carried forward to #49, not dropped:** the population is now legible, not large. A corpus-scale oracle of adjudicated `(site → correct target)` pairs is still the only thing that would make `phantom_count == 0` a statement about real repositories.

buildagent closed this issue

2026-09-06 16:32:41 +02:00

buildagent referenced this issue

2026-09-06 16:36:14 +02:00

epic: runtime plugin architecture — dynamic grammar, extraction and resolution packages #75

buildagent referenced this issue

2026-09-06 16:38:01 +02:00

gate: plugin conformance, hostile-runtime containment and production disclosure #80

buildagent referenced this issue

2026-09-06 16:45:03 +02:00

The index discloses its root but never its commit, so an agent pinned to one worktree was served another's content and could not tell #182

buildagent referenced this issue