h-dv/code-index

Watch

Fork 0

from future import … is silently absent from get_dependencies — the python walker has no arm for future_import_statement and nothing in the tree records a decision #176

New issue

Closed

opened 2026-09-06 02:40:21 +02:00 by buildagent · 2 comments

buildagent commented

2026-09-06 02:40:21 +02:00

Member

Found while authoring hand-verified benchmark questions for #51. Traced to source.

Measured

A four-line Python file:

from __future__ import annotations
import os
from collections import defaultdict

get_dependencies(...) -> imports.total: 2

Three import statements, two reported, no disclosure of the third.

On the pinned python-flask corpus this was the entire discrepancy in a hand-verified question: get_dependencies("src/flask/sansio/scaffold.py") reports imports.total: 16 against a hand-counted 17. All sixteen others are present; the only missing one is scaffold.py:1, from __future__ import annotations.

Mechanism (read at source)

tree-sitter-python parses from __future__ import X as a future_import_statement node, which is a distinct node type from import_from_statement.

crates/plugins/src/python.rs's walker has arms for import_statement and import_from_statement only. A future_import_statement matches neither, so the walk never reaches emit_from_import and no import row is produced.

Why this is filed as a defect rather than corrected in the oracle

The neighbouring case in the same batch went the other way, and the contrast is the argument. C#'s nameof(X) is also excluded from refs — but that exclusion is decided: there is a comment at the site, a test named nameof_is_not_a_call_ref, and a migration (m0028) recording it. So the benchmark oracle was corrected to match, because the tool is doing what it says.

For __future__ there is nothing. I searched the tree for __future__ and for future_import: no code path, no comment, no test, no migration, no doc line. There is no decision to defer to — only an arm that was never written. And the reply discloses no omission, while the tool promises "the imports the file declares".

Two states rendering identically: "we deliberately do not model this" and "we never saw this node type". That is the house failure mode, and the only way to tell them apart here was to grep for a decision and find none.

Why existing gates could not see it

The python plugin's extractor tests assert the import shapes they cover; future_import_statement is not among them, so nothing compares the node type to a walker arm.
corpus_ratchet pins imports as an exact count per repo. python-flask records 650, and it has been consistently 650 including every missing __future__ line since the baseline was first taken. A ratchet cannot see a row that was never emitted in any recorded run.
get_dependencies's own e2e coverage uses fixtures written by hand, and nobody writing an import fixture reaches for from __future__ import annotations.

Repro

A file containing exactly the three lines above, indexed, then get_dependencies on it. Or on the pinned python-flask corpus (sha 36e4a824f340fdee7ed50937ba8e7f6bc7d17f81): get_dependencies("src/flask/sansio/scaffold.py") → 16, against rg -c '^from |^import ' src/flask/sansio/scaffold.py → 17.

What must NOT be done to make this pass

Do not decide it silently in either direction. If __future__ should be excluded — there is a real argument, since it is a compiler directive rather than a dependency — then that needs the same three artifacts nameof has: a comment, a test, and a note. An exclusion nobody wrote down is indistinguishable from a bug, which is exactly why this issue exists.
Do not add an arm for future_import_statement alone. The finding is that a node type was missed, and the same class of miss can exist for other Python statement forms. What would close this properly is a check that the walker's arms cover the node types the grammar can produce at that position — the kind of registry this repo already uses elsewhere — rather than one more arm.
Do not re-record python-flask's imports: 650 as the way to land it. That count moving is the evidence the fix worked, and it must be blessed with a reason naming this cause, not absorbed as drift.

Measured vs inferred

The four-line fixture reading, the flask 16-vs-17, the node type, the walker's arm list, and the absence of any __future__ mention in the tree are all measured or read at source. Nothing here is inferred.

🤖 Generated with Claude Code

https://claude.ai/code/session_01K1zj5VcFJvJt3pQxe9259K

Found while authoring hand-verified benchmark questions for #51. Traced to source. ## Measured A four-line Python file: ```python from __future__ import annotations import os from collections import defaultdict ``` ``` get_dependencies(...) -> imports.total: 2 ``` Three import statements, two reported, no disclosure of the third. On the pinned `python-flask` corpus this was the entire discrepancy in a hand-verified question: `get_dependencies("src/flask/sansio/scaffold.py")` reports `imports.total: 16` against a hand-counted **17**. All sixteen others are present; the only missing one is `scaffold.py:1`, `from __future__ import annotations`. ## Mechanism (read at source) tree-sitter-python parses `from __future__ import X` as a **`future_import_statement`** node, which is a distinct node type from `import_from_statement`. `crates/plugins/src/python.rs`'s walker has arms for `import_statement` and `import_from_statement` only. A `future_import_statement` matches neither, so the walk never reaches `emit_from_import` and no import row is produced. ## Why this is filed as a defect rather than corrected in the oracle The neighbouring case in the same batch went the other way, and the contrast is the argument. C#'s `nameof(X)` is *also* excluded from refs — but that exclusion is **decided**: there is a comment at the site, a test named `nameof_is_not_a_call_ref`, and a migration (`m0028`) recording it. So the benchmark oracle was corrected to match, because the tool is doing what it says. For `__future__` there is **nothing**. I searched the tree for `__future__` and for `future_import`: no code path, no comment, no test, no migration, no doc line. There is no decision to defer to — only an arm that was never written. And the reply discloses no omission, while the tool promises "the imports the file declares". Two states rendering identically: *"we deliberately do not model this"* and *"we never saw this node type"*. That is the house failure mode, and the only way to tell them apart here was to grep for a decision and find none. ## Why existing gates could not see it - The python plugin's extractor tests assert the import shapes they cover; `future_import_statement` is not among them, so nothing compares the node type to a walker arm. - `corpus_ratchet` pins `imports` as an exact count per repo. `python-flask` records **650**, and it has been consistently 650 including every missing `__future__` line since the baseline was first taken. A ratchet cannot see a row that was never emitted in any recorded run. - `get_dependencies`'s own e2e coverage uses fixtures written by hand, and nobody writing an import fixture reaches for `from __future__ import annotations`. ## Repro A file containing exactly the three lines above, indexed, then `get_dependencies` on it. Or on the pinned `python-flask` corpus (sha `36e4a824f340fdee7ed50937ba8e7f6bc7d17f81`): `get_dependencies("src/flask/sansio/scaffold.py")` → 16, against `rg -c '^from |^import ' src/flask/sansio/scaffold.py` → 17. ## What must NOT be done to make this pass - **Do not decide it silently in either direction.** If `__future__` should be excluded — there is a real argument, since it is a compiler directive rather than a dependency — then that needs the same three artifacts `nameof` has: a comment, a test, and a note. An exclusion nobody wrote down is indistinguishable from a bug, which is exactly why this issue exists. - **Do not add an arm for `future_import_statement` alone.** The finding is that a node type was missed, and the same class of miss can exist for other Python statement forms. What would close this properly is a check that the walker's arms cover the node types the grammar can produce at that position — the kind of registry this repo already uses elsewhere — rather than one more arm. - **Do not re-record `python-flask`'s `imports: 650`** as the way to land it. That count moving is the *evidence* the fix worked, and it must be blessed with a reason naming this cause, not absorbed as drift. ## Measured vs inferred The four-line fixture reading, the flask 16-vs-17, the node type, the walker's arm list, and the absence of any `__future__` mention in the tree are all measured or read at source. Nothing here is inferred. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01K1zj5VcFJvJt3pQxe9259K

buildagent commented

2026-09-06 08:27:32 +02:00

Author

Member

FIXED, with the registry the issue asked for — and the registry immediately found a second instance of the same class in TypeScript.

The real node shape (dumped, not assumed)

(future_import_statement name: (dotted_name (identifier)) name: (dotted_name (identifier)))

stmt kind=future_import_statement  text="from __future__ import annotations, generator_stop"
    module_name field = None                     <-- NO module_name field
    child named=false kind=__future__            <-- an ANONYMOUS TOKEN
    child named=true  kind=dotted_name text="annotations"

Worth pasting because it settles a real trap: emit_from_import reused blind would read module_name as None, default the prefix to empty, and spell the module annotations. The name children (dotted_name, aliased_import) are the same shapes the from arm already handles, so the fix is a module-prefix override rather than a second emitter.

The decision, made explicitly and written down

from __future__ import annotations is a dependency the file declares. It is an import statement, get_dependencies promises "the imports the file declares", and importlib sees it. So it is emitted, with the same three artifacts nameof has — a comment at the site, a test, and this note:

from __future__ import annotations          ->  __future__.annotations
from __future__ import generator_stop as gs ->  __future__.generator_stop  alias=gs

plus an import-kind ref qualified on __future__, exactly as from collections import defaultdict already produces one qualified on collections.

The registry gate — `crates/plugins/tests/import_node_kind_registry.rs`

Rule: for each graded grammar, every NAMED, non-hidden node kind whose name carries the language's import spelling as a _-separated segment (or its plural) must carry a verdict — Reached{probe, expect}, NotAnImportStatement{why}, or KnownGap{why}.

Two properties make it more than a checklist:

Discovery is measured from the compiled grammar (Language::node_kind_count / node_kind_for_id / node_kind_is_named), so a grammar bump that introduces a new import kind fails the build rather than being noticed later.
Reachability is proven by running the real plugin.extract() on a probe and pinning the exact import rows — not by grepping the source for arm strings. That was measured, not assumed: with the arm deleted, grep -c future_import_statement python.rs still printed 4, because the comment mentions it. A grep-form registry stays green through the very mutation it exists for.

Segment-not-substring is load-bearing: rust's else_clause / where_clause and php's base_clause / catch_clause / class_interface_clause / finally_clause all contain use.

Bound, stated so it is not read as more than it is. 7 grammars (python, rust, c#, php, ts, tsx, js), 48 (grammar, kind) entries. Deliberately NOT covered, each with a written reason in the file: ruby (no import-spelled kind at all — require/load/include are method calls; asserted empty by its own test so a future grammar reddens the residual instead of hiding it); every other statement kind (the broad version of this rule is the one that gets discarded whole); whether an emitted module STRING is correct; ref rows.

The registry found a second instance of this exact defect class

TypeScript import A = require('m') (import_require_clause, ts+tsx) emits no import row at all. Same shape as this issue — a distinct grammar node kind, no walker arm, no disclosure, no decision anywhere. Registered as the one KnownGap, with a known_gaps_are_still_gaps test that goes RED when it is fixed, so promoting it is forced rather than optional. This is worth its own issue.

Also recorded as a reason rather than fixed: php class C { use T; } (trait composition) produces neither an import row nor a type ref — the trait edge is invisible to the graph. A recall finding about php.rs.

Mutations — four run, real RED

1. DATA, delete the arm → 6 passed; 3 failed

python: `future_import_statement` — extractor emitted [], registry claims ["__future__.annotations"]
the_issue_176_file_reports_every_import_line:
   left: ["collections.defaultdict", "os"]
  right: ["__future__.annotations", "collections.defaultdict", "os"]

2. PREDICATE matches NOTHING (has_import_spelling → false) → 7 passed; 2 failed

python: discovery found 0 import-spelled kinds — the predicate matches nothing
csharp: REGISTRY has a verdict for `using_directive`, which the grammar does not produce   ... x48

3. PREDICATE matches EVERYTHING (segment → contains) → 6 passed; 3 failed

rust: decoy `else_clause` was DISCOVERED as import-spelled — the rule has widened to a substring test
rust: `where_clause`; php: base_clause, catch_clause, class_interface_clause, finally_clause
ruby_has_no_import_spelled_node_kind: `in_clause` carries the spelling `use`

4. PROBE VACUITY (swap the probe for import __future__, which emits an import row but never produces the kind) → 8 passed; 1 failed

python: probe for `future_import_statement` never produces that node kind — it proves nothing

Both predicate directions were mutated, per the #178 lesson. All restores cp + md5-verified + touched; no git checkout.

Corpus deltas — MEASURED, EXPLAINED, and NOT BLESSED

COSI_CORPUS_DIR=… COSI_CORPUS_REQUIRE=1, non-vacuous (7 repos indexed):

repo	field	old → new
python-flask	`imports`	650 → 677 (+27)
python-flask	`refs`	15175 → 15202 (+27)
python-flask	`resolved`	3026 → 3026 (+0)

Every other repo unchanged by this edit — attribution proved by isolation, not asserted: with python.rs reverted to its pre-fix snapshot the ratchet reports python-flask at baseline and only cs-dapper moving, which is #172's work.

+27 is exact and explained: grep -rc "from __future__ import" over the pinned checkout = 27 lines in 27 files, one import row + one import ref each. resolved did not move — the new refs are unresolved qualified (__future__), so zero new phantoms, consistent with how every other stdlib from-import already behaves. src/flask/sansio/scaffold.py measured directly: imports.total 16 → 17, matching the hand-count.

tests/corpus/baseline.json is byte-identical. Per this lane's brief, a record that moves is a finding to report rather than something to bless, so the exact number and its cause are above and the record is untouched. This is the one thing left for the owner to action.

Second artifact this moves, also measured and also unblessed — tests/bench/ratchet.json python-flask: correct 29 → 30, recall 0.8788 → 0.9091, tool_tokens 7159 → 7192 (1.005x, inside the 0.70–1.05 band). flask.dependencies.sansio_scaffold was pinned at equals: 17 as a product defect and now returns 17 with no MISSED. Recall is a floor, so the suite stays green in the safe direction.

Gates

rustfmt --check (edition 2021) 0 · clippy -p code-index-plugins --all-targets -D warnings 0 · cargo test -p code-index-plugins 0 (270 lib + 9 registry) · RUSTDOCFLAGS="-D warnings" cargo doc 0 · doc_citation_gate 0 · precision_gate 0 — 7/7, phantoms=0 in every language · corpus ratchet 101, the +27 above, by design.

🤖 Generated with Claude Code

https://claude.ai/code/session_01K1zj5VcFJvJt3pQxe9259K

## FIXED, with the registry the issue asked for — and the registry immediately found a second instance of the same class in TypeScript. ### The real node shape (dumped, not assumed) ``` (future_import_statement name: (dotted_name (identifier)) name: (dotted_name (identifier))) stmt kind=future_import_statement text="from __future__ import annotations, generator_stop" module_name field = None <-- NO module_name field child named=false kind=__future__ <-- an ANONYMOUS TOKEN child named=true kind=dotted_name text="annotations" ``` Worth pasting because it settles a real trap: `emit_from_import` reused blind would read `module_name` as `None`, default the prefix to empty, and spell the module `annotations`. The *name* children (`dotted_name`, `aliased_import`) are the same shapes the `from` arm already handles, so the fix is a module-prefix override rather than a second emitter. ### The decision, made explicitly and written down `from __future__ import annotations` **is** a dependency the file declares. It is an import statement, `get_dependencies` promises "the imports the file declares", and `importlib` sees it. So it is emitted, with the same three artifacts `nameof` has — a comment at the site, a test, and this note: ``` from __future__ import annotations -> __future__.annotations from __future__ import generator_stop as gs -> __future__.generator_stop alias=gs ``` plus an `import`-kind ref qualified on `__future__`, exactly as `from collections import defaultdict` already produces one qualified on `collections`. ### The registry gate — `crates/plugins/tests/import_node_kind_registry.rs` **Rule:** for each graded grammar, every NAMED, non-hidden node kind whose name carries the language's import spelling **as a `_`-separated segment** (or its plural) must carry a verdict — `Reached{probe, expect}`, `NotAnImportStatement{why}`, or `KnownGap{why}`. Two properties make it more than a checklist: 1. **Discovery is measured from the compiled grammar** (`Language::node_kind_count` / `node_kind_for_id` / `node_kind_is_named`), so a grammar bump that introduces a new import kind fails the build rather than being noticed later. 2. **Reachability is proven by running the real `plugin.extract()`** on a probe and pinning the exact import rows — not by grepping the source for arm strings. That was measured, not assumed: with the arm deleted, `grep -c future_import_statement python.rs` still printed **4**, because the comment mentions it. A grep-form registry stays green through the very mutation it exists for. Segment-not-substring is load-bearing: rust's `else_clause` / `where_clause` and php's `base_clause` / `catch_clause` / `class_interface_clause` / `finally_clause` all *contain* `use`. **Bound, stated so it is not read as more than it is.** 7 grammars (python, rust, c#, php, ts, tsx, js), 48 (grammar, kind) entries. Deliberately NOT covered, each with a written reason in the file: ruby (no import-spelled kind at all — `require`/`load`/`include` are method calls; asserted empty by its own test so a future grammar reddens the residual instead of hiding it); every other statement kind (the broad version of this rule is the one that gets discarded whole); whether an emitted module STRING is correct; ref rows. ### The registry found a second instance of this exact defect class **TypeScript `import A = require('m')` (`import_require_clause`, ts+tsx) emits no import row at all.** Same shape as this issue — a distinct grammar node kind, no walker arm, no disclosure, no decision anywhere. Registered as the one `KnownGap`, with a `known_gaps_are_still_gaps` test that goes RED when it is fixed, so promoting it is forced rather than optional. **This is worth its own issue.** Also recorded as a reason rather than fixed: php `class C { use T; }` (trait composition) produces **neither an import row nor a type ref** — the trait edge is invisible to the graph. A recall finding about `php.rs`. ### Mutations — four run, real RED **1. DATA, delete the arm** → `6 passed; 3 failed` ``` python: `future_import_statement` — extractor emitted [], registry claims ["__future__.annotations"] the_issue_176_file_reports_every_import_line: left: ["collections.defaultdict", "os"] right: ["__future__.annotations", "collections.defaultdict", "os"] ``` **2. PREDICATE matches NOTHING** (`has_import_spelling` → `false`) → `7 passed; 2 failed` ``` python: discovery found 0 import-spelled kinds — the predicate matches nothing csharp: REGISTRY has a verdict for `using_directive`, which the grammar does not produce ... x48 ``` **3. PREDICATE matches EVERYTHING** (segment → `contains`) → `6 passed; 3 failed` ``` rust: decoy `else_clause` was DISCOVERED as import-spelled — the rule has widened to a substring test rust: `where_clause`; php: base_clause, catch_clause, class_interface_clause, finally_clause ruby_has_no_import_spelled_node_kind: `in_clause` carries the spelling `use` ``` **4. PROBE VACUITY** (swap the probe for `import __future__`, which emits an import row but never produces the kind) → `8 passed; 1 failed` ``` python: probe for `future_import_statement` never produces that node kind — it proves nothing ``` Both predicate directions were mutated, per the #178 lesson. All restores `cp` + md5-verified + `touch`ed; no `git checkout`. ### Corpus deltas — MEASURED, EXPLAINED, and NOT BLESSED `COSI_CORPUS_DIR=… COSI_CORPUS_REQUIRE=1`, non-vacuous (7 repos indexed): | repo | field | old → new | |---|---|---| | python-flask | `imports` | **650 → 677 (+27)** | | python-flask | `refs` | **15175 → 15202 (+27)** | | python-flask | `resolved` | **3026 → 3026 (+0)** | Every other repo unchanged by this edit — attribution proved by **isolation**, not asserted: with `python.rs` reverted to its pre-fix snapshot the ratchet reports python-flask at baseline and only `cs-dapper` moving, which is #172's work. +27 is exact and explained: `grep -rc "from __future__ import"` over the pinned checkout = **27 lines in 27 files**, one import row + one import ref each. **`resolved` did not move** — the new refs are unresolved qualified (`__future__`), so **zero new phantoms**, consistent with how every other stdlib from-import already behaves. `src/flask/sansio/scaffold.py` measured directly: `imports.total` **16 → 17**, matching the hand-count. `tests/corpus/baseline.json` is **byte-identical**. Per this lane's brief, a record that moves is a finding to report rather than something to bless, so the exact number and its cause are above and the record is untouched. **This is the one thing left for the owner to action.** Second artifact this moves, also measured and also unblessed — `tests/bench/ratchet.json` python-flask: `correct` 29 → **30**, `recall` 0.8788 → **0.9091**, `tool_tokens` 7159 → 7192 (1.005x, inside the 0.70–1.05 band). `flask.dependencies.sansio_scaffold` was pinned at `equals: 17` as a product defect and now returns 17 with no MISSED. Recall is a floor, so the suite stays green in the safe direction. ### Gates `rustfmt --check` (edition 2021) **0** · `clippy -p code-index-plugins --all-targets -D warnings` **0** · `cargo test -p code-index-plugins` **0** (270 lib + 9 registry) · `RUSTDOCFLAGS="-D warnings" cargo doc` **0** · `doc_citation_gate` **0** · `precision_gate` **0 — 7/7, phantoms=0 in every language** · corpus ratchet **101**, the +27 above, by design. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01K1zj5VcFJvJt3pQxe9259K

buildagent commented

2026-09-06 12:47:04 +02:00

Author

Member

CLOSING — fixed, and closed the way this issue asked rather than the way it forbade

Close-out lane. Verified on master 552e3a2; code read with this repo's own tools.

On master

crates/plugins/src/python.rs:266 — the missing arm exists:

self.emit_from_import(node, Some(FUTURE_MODULE))

preceded by a 25-line comment at :241-265 that dumps the future_import_statement node shape, states the decision explicitly (__future__ is a dependency — importlib can load it), and names the gate. That is the third artifact this issue demanded: a recorded decision, so the next reader is not left inferring one from an absent arm.

Graded by a registry, which is what the issue asked for instead of one more arm

crates/plugins/tests/import_node_kind_registry.rs — 9 tests, all passing (EXIT=0 here). It discovers import node kinds from the compiled grammar rather than from a hand-written list, and proves reachability by actually running plugin.extract() — so a future grammar bump that introduces another unhandled import kind fails the gate instead of going silent, which is the generalisation this issue explicitly preferred over a per-case fix.

tests/corpus/baseline.json moved and was blessed with the exact attribution — +27 / 27 lines in 27 files, resolved unmoved — rather than blessed to make something pass.

Two gaps the registry itself surfaced, which belong in new issues rather than here

Recording them so they are not lost when this closes:

TypeScript import A = require('m') — import_node_kind_registry.rs:574 registers it (import_require_clause, ts + tsx) as a KnownGap: "DOES declare a dependency on m and the extractor emits nothing — the same defect class as #176, in a second language." At least protected by known_gaps_are_still_gaps.
PHP trait composition — class C { use T; } produces neither an import row nor a type ref, so the trait edge is invisible to the graph. Protected by nothing found in the tree.

Closing.

## CLOSING — fixed, and closed the way this issue asked rather than the way it forbade Close-out lane. Verified on master `552e3a2`; code read with this repo's own tools. ### On master `crates/plugins/src/python.rs:266` — the missing arm exists: ```rust self.emit_from_import(node, Some(FUTURE_MODULE)) ``` preceded by a 25-line comment at `:241-265` that dumps the `future_import_statement` node shape, states the decision explicitly (`__future__` **is** a dependency — `importlib` can load it), and names the gate. That is the third artifact this issue demanded: a recorded decision, so the next reader is not left inferring one from an absent arm. ### Graded by a registry, which is what the issue asked for instead of one more arm `crates/plugins/tests/import_node_kind_registry.rs` — 9 tests, all passing (**EXIT=0** here). It discovers import node kinds from the **compiled grammar** rather than from a hand-written list, and proves reachability by actually running `plugin.extract()` — so a future grammar bump that introduces another unhandled import kind fails the gate instead of going silent, which is the generalisation this issue explicitly preferred over a per-case fix. `tests/corpus/baseline.json` moved and was blessed with the exact attribution — `+27 / 27 lines in 27 files`, `resolved` unmoved — rather than blessed to make something pass. ### Two gaps the registry itself surfaced, which belong in new issues rather than here Recording them so they are not lost when this closes: 1. **TypeScript `import A = require('m')`** — `import_node_kind_registry.rs:574` registers it (`import_require_clause`, ts + tsx) as a `KnownGap`: *"DOES declare a dependency on `m` and the extractor emits nothing — the same defect class as #176, in a second language."* At least protected by `known_gaps_are_still_gaps`. 2. **PHP trait composition** — `class C { use T; }` produces neither an import row nor a type ref, so the trait edge is invisible to the graph. Protected by nothing found in the tree. Closing.

buildagent closed this issue

2026-09-06 12:47:16 +02:00

buildagent referenced this issue

2026-09-06 12:51:07 +02:00

gate: plugin conformance, hostile-runtime containment and production disclosure #80

buildagent referenced this issue

2026-09-06 12:51:28 +02:00

epic: runtime plugin architecture — dynamic grammar, extraction and resolution packages #75