h-dv/code-index

Watch

Fork 0

`unexpanded_supertypes` does not hold supertypes: it holds any TYPE_POSITION ref in the container span, so a Rust enum ships its variant payload types #240

New issue

Closed

opened 2026-09-09 13:56:03 +02:00 by buildagent · 1 comment

buildagent commented

2026-09-09 13:56:03 +02:00

Member

Found by the #205 lane while bounding this field, and deliberately not folded into that fix — a behaviour change smuggled into a cap would have been unattributable, which is the same reason #205's reporter refused to ship the cap on its own.

The fact

MemberCensusBasis::unexpanded_supertypes is populated by:

any TYPE_POSITION type ref inside the container span, outside every child span

That is not what the name says, and not what the doc promises. The predicate is "a type mentioned somewhere in this container's body that no child symbol claims" — supertypes satisfy it, and so does a great deal else.

Two measured instances

Rust enums declare zero child symbols. So every type named in every variant is inside the container span and outside every child span. Expr in rust-analyzer's crates/hir-def/src/hir.rs produces 27 entries — all of them variant payload types, none of them supertypes. Before #205's cap, all 27 shipped in a safe_delete reply under a field named unexpanded_supertypes.

JavaScript static {} blocks land the same way. A new A2() inside a static initialiser block is a TYPE_POSITION ref in the container span with no child claiming it. Note this is currently depended on: the JS fixture added in #205 relies on this behaviour and says so in its own comment — it will go RED when this is fixed, which is intended and is the marker for whoever fixes it.

Why it matters beyond the name

unexpanded_supertypes is read as evidence about a hierarchy — its whole role in the census is to let a reader tell "one implicit root object" from "a real inheritance chain". A Rust enum with 27 payload types reads as a deep hierarchy and is not one. The field is not merely mis-named; it is mis-informative at exactly the question it exists to answer.

The #205 cap bounds the damage (8 entries, with unexpanded_supertypes_total saying how many were cut) and does not touch the correctness of what is in the list. That was the right split.

What must NOT be done

Do not rename the field to match the predicate. "Types mentioned in the container body" is not a fact anyone asked for, and shipping it under an honest name would just be an honestly-named field with no consumer. The census wants supertypes; the question is how to get them.
Do not filter by heuristics on the type name. Guessing which of 27 names "looks like a base class" is how phantoms get minted, and this repo has the receipts.
Do not fix it without checking what depends on the current behaviour. At minimum the #205 JS fixture does, deliberately. Grep for others before changing the predicate.
Do not fold it into an unrelated change. It moves census output on real repos; it needs its own attribution.

Shape of a fix

The extractors know the difference — a Rust supertrait bound, a Python base list, a C#/TS interface list, a PHP extends/implements, a Ruby include are all syntactically distinct from an incidental type mention. The honest fix is a ref kind or role that says "this type ref is a declared supertype", emitted by the plugins that can see it, rather than a span-containment heuristic in the daemon.

That is a wire/plugin change, so it is not small. An intermediate step worth costing: keep the span predicate but exclude containers whose language+kind declares supertypes syntactically elsewhere (a Rust enum declares none, so it should contribute none), which fixes the measured instance without a new ref kind.

What a fix must prove

A Rust enum with typed variants reports zero unexpanded supertypes, and a Rust trait with supertrait bounds still reports them.
Anti-vacuity, both arms: a container with real supertypes must not lose them, and a container with none must report a measured zero rather than an absence.
The #205 JS fixture's dependence on the current behaviour is addressed explicitly — updated with its comment, not silently.
The census delta on the corpus is read at source, per container, not counted.
All seven languages are covered by one clause, not seven arms. #205's own fixture (the deep column on CENSUS_ROWS) is the pattern.

#205 (the bounding fix, closed — this is the correctness residual it deliberately left), #199 (the census this field belongs to, and the inherited/implicit member class it interacts with).

Filed 2026-09-09 against master 5ac45d9.

Found by the #205 lane while bounding this field, and deliberately **not** folded into that fix — a behaviour change smuggled into a cap would have been unattributable, which is the same reason #205's reporter refused to ship the cap on its own. ## The fact `MemberCensusBasis::unexpanded_supertypes` is populated by: > any `TYPE_POSITION` type ref inside the container span, outside every child span That is not what the name says, and not what the doc promises. The predicate is *"a type mentioned somewhere in this container's body that no child symbol claims"* — supertypes satisfy it, and so does a great deal else. ## Two measured instances **Rust enums declare zero child symbols.** So every type named in every variant is inside the container span and outside every child span. `Expr` in `rust-analyzer`'s `crates/hir-def/src/hir.rs` produces **27 entries** — all of them variant *payload* types, none of them supertypes. Before #205's cap, all 27 shipped in a `safe_delete` reply under a field named `unexpanded_supertypes`. **JavaScript `static {}` blocks land the same way.** A `new A2()` inside a static initialiser block is a `TYPE_POSITION` ref in the container span with no child claiming it. Note this is currently *depended on*: the JS fixture added in #205 relies on this behaviour and says so in its own comment — **it will go RED when this is fixed**, which is intended and is the marker for whoever fixes it. ## Why it matters beyond the name `unexpanded_supertypes` is read as evidence about a **hierarchy** — its whole role in the census is to let a reader tell "one implicit root object" from "a real inheritance chain". A Rust enum with 27 payload types reads as a deep hierarchy and is not one. The field is not merely mis-named; it is mis-*informative* at exactly the question it exists to answer. The #205 cap bounds the damage (8 entries, with `unexpanded_supertypes_total` saying how many were cut) and does not touch the correctness of what is in the list. That was the right split. ## What must NOT be done - **Do not rename the field to match the predicate.** "Types mentioned in the container body" is not a fact anyone asked for, and shipping it under an honest name would just be an honestly-named field with no consumer. The census wants supertypes; the question is how to get them. - **Do not filter by heuristics on the type name.** Guessing which of 27 names "looks like a base class" is how phantoms get minted, and this repo has the receipts. - **Do not fix it without checking what depends on the current behaviour.** At minimum the #205 JS fixture does, deliberately. Grep for others before changing the predicate. - **Do not fold it into an unrelated change.** It moves census output on real repos; it needs its own attribution. ## Shape of a fix The extractors know the difference — a Rust supertrait bound, a Python base list, a C#/TS interface list, a PHP `extends`/`implements`, a Ruby `include` are all syntactically distinct from an incidental type mention. The honest fix is a ref kind or role that says *"this type ref is a declared supertype"*, emitted by the plugins that can see it, rather than a span-containment heuristic in the daemon. That is a wire/plugin change, so it is not small. An intermediate step worth costing: keep the span predicate but **exclude containers whose language+kind declares supertypes syntactically elsewhere** (a Rust `enum` declares none, so it should contribute none), which fixes the measured instance without a new ref kind. ## What a fix must prove - A Rust `enum` with typed variants reports **zero** unexpanded supertypes, and a Rust `trait` with supertrait bounds still reports them. - **Anti-vacuity, both arms:** a container with real supertypes must not lose them, and a container with none must report a measured zero rather than an absence. - The `#205` JS fixture's dependence on the current behaviour is addressed explicitly — updated with its comment, not silently. - The census delta on the corpus is **read at source**, per container, not counted. - All seven languages are covered by one clause, not seven arms. #205's own fixture (the `deep` column on `CENSUS_ROWS`) is the pattern. ## Related #205 (the bounding fix, closed — this is the correctness residual it deliberately left), #199 (the census this field belongs to, and the inherited/implicit member class it interacts with). Filed 2026-09-09 against `master` `5ac45d9`.

buildagent commented

2026-09-09 22:17:32 +02:00

Author

Member

Fixed on master at efe6f52 (merged 8a8ea5c). First: the headline instance in this issue is wrong, and I wrote it twice.

My error

I claimed Expr's 27 entries "shipped in a safe_delete reply" — here, and again when closing #205. They did not, and could not.

A Rust enum emits zero child symbols (measured: 0 of 825 enums in rust-analyzer), and member_census_basis only runs for a symbol whose parent is a member container. So no enum has ever been reachable through the shipped tool. The 27 strings were measured by running the predicate over every container — which is also how #205's own max 37 / over-8 34 table was taken, so that number is the same artefact.

The defect in the predicate was real and worth fixing. That particular consequence was not, and I asserted it as measured. The enum half is now graded over refs, with the reachability claim pinned so a plugin that starts emitting variants as children fails the test rather than outliving its reason.

The fix, and the measurement that rejected the cheap option

raw::ref_role::SUPERTYPE (bit 20), emitted at declared-supertype slots by all six builtin plugins and the packaged Ruby guest. The daemon predicate is (r.roles & SUPERTYPE) != 0 — one clause, seven languages.

I briefed the issue's intermediate (exclude containers whose kind declares no supertypes) as worth costing. Costed over the population the census can actually reach — a container with ≥1 child, since only such a container is somebody's parent:

repo	reachable rows	removed by kind-exclusion
rust-analyzer	9,015	46
rust-ripgrep	770	7
all other pinned repos	9,993	0

0.5%. And it cannot touch the dominant case structurally: a Rust impl Trait for T does declare a supertype, and its self type sits in the same slot — and 8,755 of rust-analyzer's 9,015 reachable rows are impl headers. It also leaves C#/PHP field annotations, generic bounds, where-clauses and trait associated types untouched.

Cost of the honest path: core bit + abi registry, 6 plugins + wasm guest (rebuilt, digest re-recorded, de.h-dv.ruby → 0.6.0), and m0065_supertype_roles (schema v65, a forced re-parse — the bit comes out of the grammar walk, so no SQL can derive it).

The corpus delta, read at source

19,893 reachable rows → 14,295. 5,598 removed, ZERO added across nine pinned repos — so containment holds empirically, not just by doc.

Every removed row adjudicated by class:

n	class	verdict
3,267	`impl Tr for T` self type or generic arg	correct
1,806	inherent `impl T` — the container's own type	correct
184	Rust where-clause / generic bound / header continuation	correct
144	C#/PHP field or property annotation	correct
78	Rust trait associated type + bound	correct
59	Rust struct generic bound	correct
18+18+18	trait param bounds; C# static field initialiser; Python `metaclass=` / C# attribute	correct — a metaclass is the class's own type, not a base
5	C# `where T : I`, TS abstract member sig, `mod`-level `type`, TS interface constraint	correct

Not one was a declared supertype. Documented residual, recorded on the constant rather than hidden: a generic argument of a declared supertype still takes the bit (Struct in impl From<Struct> for VariantDef).

tests/corpus/baseline.json md5 unmoved (4b8dad0f…), verified by me post-merge. package-baseline.json moved on conditions only — schema 64→65, ruby 0.5.0→0.6.0 — with every count./gen./rule./stage./influence. value byte-identical.

Mutations — and two that are more interesting than a RED

I re-ran M4 (widen the daemon clause back to TYPE_POSITION) myself. RED on three tests, and it reproduces the exact leak:

javascript: ... MEASURED: `extends A1`, `extends Lib.Base.Inner` and
`extends Mix(A1, ..., A12)` produce 1, 1 and 0. Before #240 this row reached
12 through the `static {}` block below ...
  unexpanded_supertypes: ["A1"..."A8"], unexpanded_supertypes_total: Some(12)

Restored by cp, md5 verified, re-run 34 passed.

Two entries worth more than their verdict:

M5 was not a mutation at all — deleting the child-span exclusion is a compile error on an unused argument. Recorded as such rather than counted, and replaced by M5b (AND 1 = 0), which compiles and is RED.
M9 went RED on the WRONG GUARD. A second row claiming the JS exemption reddened via that arm's own Some(1) assertion, which fired before the post-loop reached_the_cap count — so the count itself was ungraded. A table-level guard was added asserting the exempted set is exactly ["javascript"], and M9b is RED on it. A red that does not prove what you think is the failure mode this repo keeps finding, and catching it inside one's own new work is the good version.
M6 SURVIVED on the corpus containment loop — no non-type-position ref exists inside any Rust supertype slot, so the corpus cannot see it. A new unit test asks tp_roles directly and is RED on it. Reported as a survivor, then covered.
M7 was RED on two independently sufficient gates (the ROLE_NAMES assertion and the wasm rebuild byte-compare) and is reported as such rather than as one clean kill.

Three things this issue and #205 got wrong

The headline instance — above.
TYPE_POSITION's doc overclaims PHP. It lists "PHP use <Trait>" among its slots; probed on class Thing extends Base { use Helper; } the PHP plugin emits no ref of any kind there. Measurement recorded on the new constant; not fixed, because emitting there would add refs and move binds.
SUPERTYPE_LIST_CAP is now unexercised. Post-fix the largest declared supertype list anywhere on the pinned corpus is 5. #205's entire justification for 8 rested on a population that turns out to have been payloads and bounds — so the cap is back in exactly the state #205 refused to ship one in. Kept (the list has no structural bound and the cut is graded at 12 by fixture), but its doc no longer claims to be live on real code.

The #205 JS fixture

Went RED exactly as its own comment predicted, and is addressed explicitly rather than deleted. JavaScript extends takes one expression and has no interface list, so the row carries a recorded deep_ungradeable exemption with its measurement in it. The exempted arm is not a skip: it asserts Some(1) over a fixture that still holds the eleven new A_n() calls, so it is RED at 12 if the leak returns and RED at 0 if the one real base is lost. Both arms run.

The old child_exclusion_ungradeable column is gone by measurement, not convenience: with SUPERTYPE as the clause all seven languages can put a supertype ref inside a member's span, so its recorded reason no longer describes anything.

Unrelated finding

code-index-test-support --test fork_window failed in a concurrent run with install the stand-in: NotFound and passes in isolation — it builds its stand-in at a fixed temp_dir()/cosi-fork-window-standin/, so parallel lanes race on one binary. Not caused by this change.

Verification

fmt clean · clippy --workspace --all-targets -D warnings clean · cargo test --workspace --no-fail-fast 332/332, 0 failed · precision_gate --nocapture 7/7, phantom_count 0 with populations printed and unshrunk (re-run by me post-merge: js 1, php 3, csharp 3, ruby 3, ts 4, rust 6, python 7) · corpus_ratchet with COSI_CORPUS_REQUIRE=1 green, baseline unmoved · rustdoc -D warnings clean.

Closing.

Fixed on `master` at `efe6f52` (merged `8a8ea5c`). **First: the headline instance in this issue is wrong, and I wrote it twice.** ## My error I claimed `Expr`'s 27 entries *"shipped in a `safe_delete` reply"* — here, and again when closing #205. They did not, and could not. A Rust `enum` emits **zero child symbols** (measured: 0 of 825 enums in rust-analyzer), and `member_census_basis` only runs for a symbol whose **parent** is a member container. So no enum has ever been reachable through the shipped tool. The 27 strings were measured by running the predicate over *every* container — which is also how #205's own `max 37 / over-8 34` table was taken, so that number is the same artefact. The defect in the predicate was real and worth fixing. That particular consequence was not, and I asserted it as measured. The enum half is now graded over **refs**, with the reachability claim pinned so a plugin that starts emitting variants as children fails the test rather than outliving its reason. ## The fix, and the measurement that rejected the cheap option `raw::ref_role::SUPERTYPE` (bit 20), emitted at declared-supertype slots by all six builtin plugins and the packaged Ruby guest. The daemon predicate is `(r.roles & SUPERTYPE) != 0` — one clause, seven languages. I briefed the issue's intermediate (exclude containers whose kind declares no supertypes) as worth costing. **Costed over the population the census can actually reach** — a container with ≥1 child, since only such a container is somebody's parent: | repo | reachable rows | removed by kind-exclusion | |---|---:|---:| | rust-analyzer | 9,015 | **46** | | rust-ripgrep | 770 | **7** | | all other pinned repos | 9,993 | **0** | **0.5%.** And it cannot touch the dominant case *structurally*: a Rust `impl Trait for T` **does** declare a supertype, and its self type sits in the same slot — and 8,755 of rust-analyzer's 9,015 reachable rows are `impl` headers. It also leaves C#/PHP field annotations, generic bounds, where-clauses and trait associated types untouched. Cost of the honest path: core bit + abi registry, 6 plugins + wasm guest (rebuilt, digest re-recorded, `de.h-dv.ruby` → 0.6.0), and `m0065_supertype_roles` (schema v65, a forced re-parse — the bit comes out of the grammar walk, so no SQL can derive it). ## The corpus delta, read at source **19,893 reachable rows → 14,295. 5,598 removed, ZERO added** across nine pinned repos — so containment holds empirically, not just by doc. Every removed row adjudicated by class: | n | class | verdict | |---:|---|---| | 3,267 | `impl Tr for T` self type or generic arg | correct | | 1,806 | inherent `impl T` — the container's own type | correct | | 184 | Rust where-clause / generic bound / header continuation | correct | | 144 | C#/PHP field or property annotation | correct | | 78 | Rust trait associated type + bound | correct | | 59 | Rust struct generic bound | correct | | 18+18+18 | trait param bounds; C# static field initialiser; Python `metaclass=` / C# attribute | correct — a metaclass is the class's own type, not a base | | 5 | C# `where T : I`, TS abstract member sig, `mod`-level `type`, TS interface constraint | correct | **Not one was a declared supertype.** Documented residual, recorded on the constant rather than hidden: a generic *argument of* a declared supertype still takes the bit (`Struct` in `impl From<Struct> for VariantDef`). `tests/corpus/baseline.json` md5 **unmoved** (`4b8dad0f…`), verified by me post-merge. `package-baseline.json` moved on **conditions only** — schema 64→65, ruby 0.5.0→0.6.0 — with every `count.`/`gen.`/`rule.`/`stage.`/`influence.` value byte-identical. ## Mutations — and two that are more interesting than a RED I re-ran **M4** (widen the daemon clause back to `TYPE_POSITION`) myself. RED on three tests, and it reproduces the exact leak: ``` javascript: ... MEASURED: `extends A1`, `extends Lib.Base.Inner` and `extends Mix(A1, ..., A12)` produce 1, 1 and 0. Before #240 this row reached 12 through the `static {}` block below ... unexpanded_supertypes: ["A1"..."A8"], unexpanded_supertypes_total: Some(12) ``` Restored by `cp`, md5 verified, re-run 34 passed. Two entries worth more than their verdict: - **M5 was not a mutation at all** — deleting the child-span exclusion is a *compile* error on an unused argument. Recorded as such rather than counted, and replaced by M5b (`AND 1 = 0`), which compiles and is RED. - **M9 went RED on the WRONG GUARD.** A second row claiming the JS exemption reddened via that arm's own `Some(1)` assertion, which fired *before* the post-loop `reached_the_cap` count — so the count itself was **ungraded**. A table-level guard was added asserting the exempted set is exactly `["javascript"]`, and M9b is RED on it. A red that does not prove what you think is the failure mode this repo keeps finding, and catching it inside one's own new work is the good version. - **M6 SURVIVED** on the corpus containment loop — no non-type-position ref exists inside any Rust supertype slot, so the corpus cannot see it. A new unit test asks `tp_roles` directly and is RED on it. Reported as a survivor, then covered. - **M7 was RED on two independently sufficient gates** (the `ROLE_NAMES` assertion *and* the wasm rebuild byte-compare) and is reported as such rather than as one clean kill. ## Three things this issue and #205 got wrong 1. **The headline instance** — above. 2. **`TYPE_POSITION`'s doc overclaims PHP.** It lists "PHP `use <Trait>`" among its slots; probed on `class Thing extends Base { use Helper; }` the PHP plugin emits **no ref of any kind** there. Measurement recorded on the new constant; not fixed, because emitting there would add refs and move binds. 3. **`SUPERTYPE_LIST_CAP` is now unexercised.** Post-fix the largest declared supertype list anywhere on the pinned corpus is **5**. #205's entire justification for 8 rested on a population that turns out to have been payloads and bounds — so the cap is back in exactly the state #205 refused to ship one in. Kept (the list has no structural bound and the cut is graded at 12 by fixture), but its doc no longer claims to be live on real code. ## The #205 JS fixture Went RED exactly as its own comment predicted, and is addressed explicitly rather than deleted. JavaScript `extends` takes one expression and has no interface list, so the row carries a recorded `deep_ungradeable` exemption **with its measurement in it**. The exempted arm is not a skip: it asserts `Some(1)` over a fixture that **still holds the eleven `new A_n()` calls**, so it is RED at 12 if the leak returns and RED at 0 if the one real base is lost. Both arms run. The old `child_exclusion_ungradeable` column is **gone by measurement**, not convenience: with `SUPERTYPE` as the clause all seven languages can put a supertype ref inside a member's span, so its recorded reason no longer describes anything. ## Unrelated finding `code-index-test-support --test fork_window` failed in a concurrent run with `install the stand-in: NotFound` and passes in isolation — it builds its stand-in at a fixed `temp_dir()/cosi-fork-window-standin/`, so parallel lanes race on one binary. Not caused by this change. ## Verification `fmt` clean · `clippy --workspace --all-targets -D warnings` clean · `cargo test --workspace --no-fail-fast` **332/332, 0 failed** · `precision_gate --nocapture` **7/7, phantom_count 0** with populations printed and unshrunk (re-run by me post-merge: js 1, php 3, csharp 3, ruby 3, ts 4, rust 6, python 7) · `corpus_ratchet` with `COSI_CORPUS_REQUIRE=1` green, baseline unmoved · rustdoc `-D warnings` clean. Closing.