Resolver precision audit on v0.41.48: 43 wrong binds in 983 sampled, in four evidence classes

buildagent commented

2026-11-10 17:49:53 +01:00

Member

A read-only audit of the released binary over the nine pinned corpus repos (331,752 resolved references). A codex worker took a deterministic sample per (language, rule), 983 binds, and judged each from the reference site and the target. I read the wrong-bind list; I did not re-judge the sample.

Result: 43 wrong, 1 undecidable. Controls on strong-evidence rules: 120 of 120 correct.

Caveat on the "correct" side: the judging marked ranges correct unless a row was flagged, so 43 is a floor on the wrong binds, not a precision estimate. The counts are not population-weighted.

The four classes, by the evidence that would have refused them

1. Absolute or global names (18). A name written as global binds to a namespaced project symbol of the same name.

PHP \curl_share_init() → a namespaced test shim; new \InvalidArgumentException → the project's own subclass. Rule 11: 0 of 15 correct on php-guzzle.
Ruby ::Logger → Sinatra::Middleware::Logger. Rule 11: 0 of 3.

2. Receiver identity (17). A member call on a receiver whose type is not a project type binds by name to a project method.

Python content_type.encode("ascii") → SessionBase.encode; apps.ready → an AppConfig.ready in a file whose stem is apps.
Ruby x.split(",").map → Sinatra::Streaming#map.
Rust syntax.clone() → Parse<T>::clone.
TypeScript z.readonly(x) (namespace factory) → the instance method readonly().

3. Lexical and file isolation (5). The target is a binding no other file can see, or the site is a definition.

JS exports.request = req → var request in test/exports.js; app.render = function render → a helper in test/app.engine.js. Both sites define a member; rules 40/42 anchored the qualifier by FILE STEM.
C# pattern variable e → a parameter in another file.

4. Exact module or type path (4, Rust). Unqualified Result → a private one-parameter alias in another file; ast::Item → an associated type Item.

Rules involved

Rule 11 (qualified flag plus a unique name, no check of what the qualifier says) and rule 43 (pure-relative qualifier plus same-package name) bind across files with no target-specific evidence; together 0.66% of resolved references. The qualifier tiers 40/42 anchor by file stem. The name tiers 12, 13 and 30 supply the receiver-identity cases.

Plan for the current train

Class 1: one rule for absolute-qualified names in every language that has them (PHP \, Ruby ::, C# global::), extending the PHP work already in progress for #531.
Class 3 and the stem-anchored part of class 2: a cross-file bind needs a target visible outside its file, and a qualifier is anchored to a file only through an import or binding that names it, never through a matching stem. Same defect family as the importer list in #543.
Class 2 proper is the evidence-less member rule of #531, disabled by operator decision until receiver facts exist. This audit is new evidence for that decision; it is not changed here.
Class 4 goes with #541 (Rust scope).

Every change ships with a bind account against this release: removals shown wrong, additions shown correct.

Related: #531, #538, #541, #543.

🤖 Generated with Claude Code

A read-only audit of the released binary over the nine pinned corpus repos (331,752 resolved references). A codex worker took a deterministic sample per (language, rule), 983 binds, and judged each from the reference site and the target. I read the wrong-bind list; I did not re-judge the sample. **Result: 43 wrong, 1 undecidable. Controls on strong-evidence rules: 120 of 120 correct.** Caveat on the "correct" side: the judging marked ranges correct unless a row was flagged, so 43 is a floor on the wrong binds, not a precision estimate. The counts are not population-weighted. ## The four classes, by the evidence that would have refused them **1. Absolute or global names (18).** A name written as global binds to a namespaced project symbol of the same name. - PHP `\curl_share_init()` → a namespaced test shim; `new \InvalidArgumentException` → the project's own subclass. Rule 11: 0 of 15 correct on php-guzzle. - Ruby `::Logger` → `Sinatra::Middleware::Logger`. Rule 11: 0 of 3. **2. Receiver identity (17).** A member call on a receiver whose type is not a project type binds by name to a project method. - Python `content_type.encode("ascii")` → `SessionBase.encode`; `apps.ready` → an `AppConfig.ready` in a file whose stem is `apps`. - Ruby `x.split(",").map` → `Sinatra::Streaming#map`. - Rust `syntax.clone()` → `Parse<T>::clone`. - TypeScript `z.readonly(x)` (namespace factory) → the instance method `readonly()`. **3. Lexical and file isolation (5).** The target is a binding no other file can see, or the site is a definition. - JS `exports.request = req` → `var request` in `test/exports.js`; `app.render = function render` → a helper in `test/app.engine.js`. Both sites define a member; rules 40/42 anchored the qualifier by FILE STEM. - C# pattern variable `e` → a parameter in another file. **4. Exact module or type path (4, Rust).** Unqualified `Result` → a private one-parameter alias in another file; `ast::Item` → an associated type `Item`. ## Rules involved Rule 11 (qualified flag plus a unique name, no check of what the qualifier says) and rule 43 (pure-relative qualifier plus same-package name) bind across files with no target-specific evidence; together 0.66% of resolved references. The qualifier tiers 40/42 anchor by file stem. The name tiers 12, 13 and 30 supply the receiver-identity cases. ## Plan for the current train - Class 1: one rule for absolute-qualified names in every language that has them (PHP `\`, Ruby `::`, C# `global::`), extending the PHP work already in progress for #531. - Class 3 and the stem-anchored part of class 2: a cross-file bind needs a target visible outside its file, and a qualifier is anchored to a file only through an import or binding that names it, never through a matching stem. Same defect family as the importer list in #543. - Class 2 proper is the evidence-less member rule of #531, disabled by operator decision until receiver facts exist. This audit is new evidence for that decision; it is not changed here. - Class 4 goes with #541 (Rust scope). Every change ships with a bind account against this release: removals shown wrong, additions shown correct. Related: #531, #538, #541, #543. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

buildagent commented

2026-11-10 19:49:52 +01:00

Author

Member

Second, independent judge on the "correct" side of this audit.

The first auditor marked ranges correct unless a row was flagged, so the 939 correct verdicts were lenient. A different model (Gemini 4 Carbon) re-judged 140 of them from source without seeing the first reasons: 20 per language, name-based rules only (10–16, 20, 30, 40–44), selected by hash and saved before judging.

Result: 140 of 140 confirmed, 0 overturned, 0 undecidable. Each verdict carries one line of file:line evidence. I checked two citations against the source and both were exact, including a non-obvious one: Ruby autoload :Extension in Sinatra::Contrib really is Sinatra::Contrib::Loader#autoload, through extend Loader.

What this supports: on this sample the first audit's correct side holds, so the 43 wrong binds are not the tip of a much larger undercount. With 0 of 140, the false-correct rate among those verdicts is unlikely to exceed about 2%. It says nothing new about the 43 themselves or about rules that were not sampled.

🤖 Generated with Claude Code

Second, independent judge on the "correct" side of this audit. The first auditor marked ranges correct unless a row was flagged, so the 939 correct verdicts were lenient. A different model (Gemini 4 Carbon) re-judged 140 of them from source without seeing the first reasons: 20 per language, name-based rules only (10–16, 20, 30, 40–44), selected by hash and saved before judging. **Result: 140 of 140 confirmed, 0 overturned, 0 undecidable.** Each verdict carries one line of file:line evidence. I checked two citations against the source and both were exact, including a non-obvious one: Ruby `autoload :Extension` in `Sinatra::Contrib` really is `Sinatra::Contrib::Loader#autoload`, through `extend Loader`. What this supports: on this sample the first audit's correct side holds, so the 43 wrong binds are not the tip of a much larger undercount. With 0 of 140, the false-correct rate among those verdicts is unlikely to exceed about 2%. It says nothing new about the 43 themselves or about rules that were not sampled. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

buildagent referenced this issue

2026-11-11 07:24:31 +01:00

v0.41.49: PHP namespace paths, settled answers, validated configuration, rules files removed #567

buildagent referenced this issue from a commit

2026-11-11 16:41:02 +01:00

Merge codex-r3: PHP namespace paths, parent:: and global-root calls (#531, #566 class 1)

buildagent referenced this issue from a commit

2026-11-11 16:41:02 +01:00

Merge codex-r2 (importer part): script imports resolved by path, exported constants as import targets (#543)

buildagent commented

2026-11-11 17:50:15 +01:00

Author

Member

Status with v0.41.49 (released, PHP package 0.1.6 bundled)

PHP qualified calls and constructors now resolve by the written namespace path, the imported class identity and the namespace\ marker; a name with a leading backslash binds only to the global symbol it names.
Bind account against v0.41.48 on the nine pinned repositories: only php-guzzle moved. 55 binds removed (all audited wrong), 57 added and 5 retargeted (all audited correct), no correct bind lost.

Stays open for: Ruby ::Logger (three audited wrong targets), the receiver-less member binds of the audit, the Rust exact module/type path rows, and the file-isolation rules. Those rules are built (branch codex-r2) and removed 19 wrong binds without losing a correct one, but their additions and two integration checks are not yet qualified, so they are deferred to the next train.

**Status with v0.41.49** (released, PHP package 0.1.6 bundled) - PHP qualified calls and constructors now resolve by the written namespace path, the imported class identity and the `namespace\` marker; a name with a leading backslash binds only to the global symbol it names. - Bind account against v0.41.48 on the nine pinned repositories: only php-guzzle moved. 55 binds removed (all audited wrong), 57 added and 5 retargeted (all audited correct), no correct bind lost. **Stays open for:** Ruby `::Logger` (three audited wrong targets), the receiver-less member binds of the audit, the Rust exact module/type path rows, and the file-isolation rules. Those rules are built (branch `codex-r2`) and removed 19 wrong binds without losing a correct one, but their additions and two integration checks are not yet qualified, so they are deferred to the next train.

buildagent referenced this issue from a commit

2026-11-12 05:46:02 +01:00

Merge codex-r23: module-evidence statements probe by file key (#566 cost)

buildagent referenced this issue from a commit

2026-11-12 05:46:02 +01:00

test(corpus): bless content, cost and stage baselines for the isolation rules (#566)

buildagent referenced this issue from a commit

2026-11-12 05:46:02 +01:00

corpus: re-record the tier-3 baseline after the #566 bind account

buildagent referenced this issue from a commit

2026-11-12 05:46:02 +01:00

test(javascript): the declared unbound-by-scope count on js-express is 41

buildagent referenced this issue

2026-11-12 05:46:24 +01:00

v0.41.50: module evidence for qualified references, one argument contract, one extraction census #574

buildagent referenced this issue from a commit

2026-11-12 11:04:35 +01:00

fix: require visible targets and module evidence for cross-file qualifiers

buildagent referenced this issue from a commit

2026-11-12 11:04:35 +01:00

Merge codex-r20: file-isolation rules on positive contradictions (#566)

buildagent commented

2026-11-12 11:41:27 +01:00

Author

Member

Released in v0.41.50.

A qualified reference is no longer bound across files on name evidence when the file's own imports or module declarations contradict the candidate; where the module evidence names one candidate, it is bound.
Against v0.41.49 on the nine pinned repositories: 19 binds removed (all wrong before), 228 added, 1 retargeted, 133 with a changed rule tag only. Every candidate addition was audited; one wrong one was removed by a rule fix before the merge. A blind second audit of 65 rows agrees on every row.
By repository (removed / added): rust-analyzer 15 / 194, rust-ripgrep 0 / 18, ts-zod 0 / 13, js-express 2 / 3, php-guzzle 2 / 0.
Every package-supplied input of the new rules passes a capability grant (13 admissions); a test shows a package without reachability_anchor cannot disambiguate a reference through its imports.
Cost: cold-index VM steps +0.7 % to +1.7 % on seven repositories, lower on two.

Still open: the 33 wrong binds listed above that this change does not touch (31 inherited, 2 found as audit controls).

Released in v0.41.50. - A qualified reference is no longer bound across files on name evidence when the file's own imports or module declarations contradict the candidate; where the module evidence names one candidate, it is bound. - Against v0.41.49 on the nine pinned repositories: 19 binds removed (all wrong before), 228 added, 1 retargeted, 133 with a changed rule tag only. Every candidate addition was audited; one wrong one was removed by a rule fix before the merge. A blind second audit of 65 rows agrees on every row. - By repository (removed / added): rust-analyzer 15 / 194, rust-ripgrep 0 / 18, ts-zod 0 / 13, js-express 2 / 3, php-guzzle 2 / 0. - Every package-supplied input of the new rules passes a capability grant (13 admissions); a test shows a package without `reachability_anchor` cannot disambiguate a reference through its imports. - Cost: cold-index VM steps +0.7 % to +1.7 % on seven repositories, lower on two. Still open: the 33 wrong binds listed above that this change does not touch (31 inherited, 2 found as audit controls).

buildagent referenced this issue

2026-11-12 18:41:44 +01:00

v0.41.51: bounded text arguments, truthful path and startup answers, review grading by effective visibility #580

buildagent referenced this issue

2026-11-12 22:22:53 +01:00

Resolved references can name unrelated methods, global names and function-local types #531

buildagent commented

2026-11-12 22:23:06 +01:00

Author

Member

Merged into #531.
PHP global paths and import/module contradiction rules shipped; see global_roots.rs and qualifier_scope.rs regression tests.
Remaining: Receiver identity; Namespace and lexical identity. The residual audit needs row-level disposition.

Merged into #531. PHP global paths and import/module contradiction rules shipped; see global_roots.rs and qualifier_scope.rs regression tests. Remaining: Receiver identity; Namespace and lexical identity. The residual audit needs row-level disposition.

buildagent closed this issue

2026-11-12 22:23:07 +01:00

buildagent referenced this issue

2026-11-13 06:15:54 +01:00

Resolved references can name unrelated methods, global names and function-local types #531

Rows
Columns

Resolver precision audit on v0.41.48: 43 wrong binds in 983 sampled, in four evidence classes #566

The four classes, by the evidence that would have refused them

Rules involved

Plan for the current train