Python calls through ANY receiver to a module-level function are unbindable by construction — method_call draws only from kind='method', and python.rs mints no module symbol at all #175
Labels
No labels
code-review
correctness
dos
performance
security
severity/high
severity/low
severity/medium
tech-debt
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
h-dv/code-index#175
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Found while authoring hand-verified benchmark questions for #51 on the pinned
python-flaskcorpus (sha36e4a824f340fdee7ed50937ba8e7f6bc7d17f81).Measured
559 unresolved refs against a single name, in a repository of 216 files. The shape is:
flaskhere names a directory package, not a module file. It gets no module symbol, so the receiver binds to nothing, and every attribute call through it is lost.This is not an exotic style. It is flask's own dominant test-call convention, which is why one name accounts for 559 refs.
It costs two different tools, and that is what makes it more than a recall number
Three hand-verified benchmark questions failed on this one gap, through two unrelated surfaces:
find_callersmisses the row —flask.send_from_directory(…)attests/test_helpers.py:89andflask.stream_template_string(…)attests/test_templating.py:36are hand-verified call sites that come back empty.change_impact/context_packmiss the edge —context_pack(send_from_directory)returnstest_role_files: ["tests/test_blueprints.py"]and nottests/test_helpers.py. The pack reachestest_blueprints.pytransitively throughsend_static_file; the only direct edge totest_helpers.pyis line 89, and it does not exist in the graph.So "which tests cover this?" — one of the five question shapes #51 names — is answered wrongly for any Python symbol whose tests use the package-qualified style. The answer is not empty, which would at least look suspicious; it is a shorter, plausible list.
Mechanism — measured symptom, inferred cause
Measured:
resolution_gapsclassifies these asref_kind: "type",reason: "no_candidate", against the nameflask. A directory package has no module symbol in the index.Inferred, not traced: that the fix belongs at the point where a package directory (a directory containing
__init__.py) could mint a module symbol keyed by the package name, so the receiver has something to bind to. I did not read the extractor or the resolver arm, and I am flagging that rather than presenting a diagnosis I did not verify. The 559 and the three failing questions are the finding; the remedy is a suggestion.Why existing gates could not see it
precision_gate's python oracle grades phantoms and per-site recall on a hand-built fixture; a package-qualified call through a directory package is not one of its probes.corpus_ratchetpinsresolvedas an exact count onpython-flask. These refs are consistently unresolved, so the count is stable and the ratchet has been green over all 559 of them since the day it was recorded.resolution_gapshas held this fact, by name and with the count, the whole time. Nothing cross-checks a diagnostic against a caller-side answer — which is the same structural gap as the one filed for overload ambiguity.Repro
Minimal form: a package directory
p/with__init__.pydefiningf(), and a siblingimport p+p.f().What must NOT be done to make this pass
flask.send_from_directoryresolving to anysend_from_directoryanywhere is a phantom generator, and Python's stdlib and third-party surface is full of shared attribute names.phantom_count == 0is the gate that must not move.find_callerswithout fixing the graph edge. Two of the three failing questions arechange_impact/context_pack. A fix that lands only in the ref-count path would close one question and leave "which tests cover this" still answering with a shorter, plausible, wrong list — which is the worse failure of the two.python-flaskcorpusresolvedcount as the success criterion. That number will move when this is fixed and it should; but the thing to assert is the three benchmark questions, because a count moving does not say which binds arrived.tests/bench/oracle/python-flask.jsonalready carries the hand-verified answers and two of them are pinned withknown_defect+currently_returnsprecisely so that a fix reports itself rather than being absorbed.🤖 Generated with Claude Code
https://claude.ai/code/session_01K1zj5VcFJvJt3pQxe9259K
Verdict: PARTIALLY REFUTED at the mechanism, and the correct half is not the half that costs recall. Fixed the disclosure; the bind is NOT safely fixable in one pass and is left NAMED.
Traced at source rather than re-derived. Measured vs inferred is marked per claim.
What the issue got right
flasknames a directory package and gets no module symbol. True — and understated:crates/plugins/src/python.rscontains zeroSymbolKind::MODULE. No Python file and no Python directory mints a module symbol, ever.project_overview().count_basis.kinds_without_use_channellistsmodulefor rust/csharp/php/ruby and not for python, andfile_outlineon apkg/__init__.pyreturnstotal: 0— the product's own channels prove the negative.What it got wrong, and it matters for where a fix would be aimed
name: "flask", ref_kind: "type", and the only ref bearing that name and kind is the receiver head fromemit_receiver_type_refs_d(python.rs:1076-1088). The call refs are namedsend_from_directoryetc., kindmethod_call, and land in a different bucket under a different reason —receiver_unbound, notno_candidate(crates/daemon/src/graph.rs:1552-1565). Measured on the pinned checkout: 704flask.<name>occurrences, of which 296 are call-shaped. The upper bound on missed call edges is 296, and real missed in-project edges are a small handful. The oracle's own triage text already saysreceiver_unboundcorrectly..pyfiles; 43 mentionflask.at all.known_defect+currently_returns" — there is ONE.grep -c '"known_defect"' tests/bench/oracle/python-flask.json→ 1. The siblingflask.who_calls.stream_template_stringsays in its ownverificationwhy it is unpinnable:agent_task_bench.rs:296-301refuses an emptycurrently_returns, so a total miss is structurally unpinnable in this harness. That is a gap in the ratchet worth its own issue — the loudest failures are the ones it cannot pin.refs.qualified = 1(crates/indexer/src/index.rs:4972) andpython.rs:781hardcodesqualified: false. The tier is never asked. And even granting a bind,send_from_directoryis defined insrc/flask/helpers.py:543;src/flask/__init__.py:20is a re-export, and no package re-export relation exists in the resolver at all.The real root cause, one sentence
Python emits
mod.func()asmethod_call;method_callrefs draw candidates only fromkind='method'(index.rs:3803, enforced identically in tier 1Q atindex.rs:5629, "ONE RULE, EVERY TIER"); and a Python module-leveldefiskind='function'. So a call through ANY receiver to ANY free function is unbindable in Python by construction — package directory or not. The directory/file distinction changes nothing about the missed edge.Why minting the module symbol alone would have been the WRONG thing to ship
It would move
refs_resolvedby ~559 and close zero of the three graded questions, while feeding the relationindex.rs:3890-3897names in its own comment as "the relation the 141 phantom edges travelled on" — lang-blind by construction, and the rule would apply to everyutils/,core/,models/directory in every indexed Python repo. A number that looks like progress on the exact issue it does not fix, paid for out of the phantom budget.WHAT SHIPPED — the honest interim, and it is a real correctness fix
The investigation found a second, independent honesty defect riding on the same root cause, and this one is fixable in isolation with zero phantom exposure:
send_from_directoryshipsname_fallback_count: 0— the pair this project's docs call TIGHT — on a symbol with a live unresolved reference in the index.local_index.rs's free-function fallback arm excludesmethod_calloutright, on a comment that names its own assumption: "a qualifiedmod::func()is already kind='call'". That holds for Rust and PHP. It is false for Python (python.rs:771emitsMETHOD_CALL).This is the same mechanism as #173, so both are fixed by one clause — the complement of the counting predicate within its own population:
Same-name, same-language, unresolved, call-shaped refs that the call-shape filter behind
name_fallback_countREMOVED. A non-zero there means the0beside it is a fact about the FILTER, not about the symbol. It is explicitly not recall — those rows are not references to this symbol, andresolution_gapsholds their reasons. Nothing is bound, nothing is fuzzy-matched,phantom_count == 0is untouched.New e2e
crates/mcp-server/tests/shape_excluded_fallback_e2e.rsgrades both this issue's shape and #173's in one fixture, with a control per language and a page-level discrimination assertion. Measured: python free function called aspkgmod.send_thing()→name_fallback_shape_excluded: 1; the bare-called control →0.Mutations run, real RED:
left: Some(0), right: Some(1)/Some(4)).>= 0formulation of the assertions was also run and passes, which is exactly why the shipped assertions pin exact counts.NAMED RESIDUALS — not fixed, and each is a real thing
self.replies.send(..)shape,index.rs:3538-3542). Honest sequencing if attempted: (i) mint python module symbols for files and package dirs and measure phantoms on all nine corpora before anything consumes them; (ii) build a package re-export relation from__init__.py; (iii) only then consider a language-conditional method-pool rule, behind its own phantom measurement.pool_class = POOL_METHOD+s.kind = 'method'andrefs.qualified = 0, not "a package directory gets no module symbol". Otherwise the fix goes to the wrong file.name_fallback_shape_excludedisOption, so an older daemon simply omits it. It has no explicit*_unavailablestamp; the field doc says absent means NOT REPORTED. Weaker than this repo's usual two-way shim, and named as such.Dogfood findings about our own tools
resolution_gapscannot reach the corpus.resolution_gaps(project: "python-flask")→project_not_found; there is no.code-index.tomland no linked projects. The number this issue is built on is unreproducible through the product's own tools — onlyagent_task_benchcan produce it. Worth considering whether corpus checkouts should be linkable.search_textcannot enumerate occurrences within one large file.matches_in_file.linescaps at 20 withlines_truncated: true(honest), butcursorpages FILES, and pinningpath_globto the single file does not help. Two sub-lanes independently fell back togrep -nfor the same audit shape ("everychild_by_field_name("name")in this file", "everyqual_scandssite inindex.rs"). Amatches_in_filecursor would close it.🤖 Generated with Claude Code
https://claude.ai/code/session_01K1zj5VcFJvJt3pQxe9259K
page_name_fallback: 0with no shape-excluded sibling #173phantom_count == 0bounds ~50 hand-written probes, not the corpus, and it is cited across the tree as an absolute guarantee #188STAYING OPEN — the bind is untouched; the issue must be restated against the traced mechanism before anyone aims code at it
Close-out lane, master
552e3a2. Title corrected, because as filed it points at the wrong file.What shipped
Only the disclosure half:
a_python_free_function_called_through_a_namespace_reports_the_exclusionpasses in both legs, andname_fallback_shape_excludednow ships on the symbol row. Record that as partial progress so it is not redone.The bind is not fixed, and both halves of the traced cause are still true on master
SymbolKind::MODULEappears inruby.rs,php.rs,csharp.rsandrust.rs— and incrates/plugins/src/python.rsnot at all. No Python file and no Python package directory mints a module symbol.crates/indexer/src/index.rs:3744-3749mapsrefs.kind = 'method_call'toPOOL_METHOD, under a comment headed "ONE RULE, EVERY TIER, INCLUDING 1Q". Python emitspkg.f()asmethod_call; a module-leveldefiskind='function'. The candidate pool cannot contain the target.So the package-directory framing in the current body is a symptom. The cause is that a Python call through any receiver to a module-level function is unbindable by construction. Aiming a fix at package directories would leave that intact.
Three measured claims in the body are wrong and should be corrected
ref_kind: "type"), not the calls — 296 of 704flask.<name>occurrences are call-shaped..pyfiles, not 216.known_defect, not two.Corrected scope
Retitle as above and replace the Mechanism section with the two facts above. The fix needs the three-step sequence the implementing lane named, each step behind its own phantom measurement — it is not a one-clause change, and the pool rule is load-bearing for every language, so widening it is exactly the kind of change that must be measured bind-for-bind before it lands.
Python calls through a package namespace never resolve — 559 on flask alone, and it costs find_callers, change_impact and context_pack togetherto Python calls through ANY receiver to a module-level function are unbindable by construction —method_calldraws only fromkind='method', and python.rs mints no module symbol at allSTAYS OPEN, and #69 shipping does NOT touch it — measured, with the second independent blocker now named
Resolver-recall lane, worktree off master
fc329a8. I own the resolver this round, so the first question was whether #69's tier-1R widening reaches this. It does not, and here is why, from the code that shipped.Measured, before and after the #69 change, on the pinned
python-flaskfc329a8)name='flask',kind='type', unresolved (the "559")method_callwithqualifier='flask'tests/test_helpers.py:89send_from_directorytarget_idByte-identical. #69's widening admits
read/writerows into tier 1R; these aremethod_callrows, and they were already in tier 1R's input before #69 and already failed it.There are TWO independent blockers, not one, and the issue names neither correctly
The previous comment traced the pool rule. That is real, and I confirm it:
send_from_directoryiskind = 'function'(src/flask/helpers.py), whilerefs.kind = 'method_call'maps toPOOL_METHOD, which admitskind = 'method'only — the "ONE RULE, EVERY TIER" comment atcrates/indexer/src/index.rs. That blocker stands.But tier 1R never gets that far, and this is the half worth adding, because it changes where a fix would have to start. Tier 1R keys on
temp.recv_binds, which is built fromrefs.kind = 'binding':import flaskproduces an IMPORT row, never a binding row. So the receiverflaskhas no proven type,recv_boundis empty for it, and the candidate join — pool rule or not — is never reached. Widening the pool alone would therefore change nothing here: a fix has to establish a TYPE for the receiver first, and that is what "python mints no module symbol" is really about.That ordering matters for anyone scoping this: the three-step sequence in the earlier comment is right, and step (i) — mint the module symbol — is not optional groundwork before the pool question, it is the blocker that fires first.
The three corrected body claims, re-measured independently
SELECT COUNT(*) FROM files WHERE lang='python'-> 83, not 216.SELECT COUNT(*) FROM symbols WHERE kind='module' AND lang='python'-> 0. Confirmed: no Python file and no Python package directory mints a module symbol.kind='type'). The call-shaped population against that receiver is 283 unresolvedmethod_callrows, which is the honest upper bound on missed call edges.Verdict
Not fixed, not attemptable as one clause, and not reachable from #69's mechanism. Retitle and restate as the earlier comment says; add the
recv_bindsfact above to the Mechanism section, because a reader who fixes only the pool rule will measure no change and conclude the diagnosis was wrong.internal_error, not the documentedpath_outside_known_roots#201Fixed on
masterat6f8cddf(mergedbb214f1). The three-step sequence in this issue is refuted by construction, and the fix needed none of it.What was actually blocking it
This issue says "step (i) mint the module symbol … is the blocker that fires first", and that "a fix has to establish a TYPE for the receiver first".
Neither is true. The blocker was that the plugin called a module-qualified call a
method_call.import Xis the only Python statement that provably binds a module, sopython.rsnow emits a call through such a name as a qualifiedcall— the rule Rust and PHP already follow formod::func(). Once it stops being receiver-mode,recv_bindsis irrelevant and the existing tier-1Q anchor answers.No module symbols minted. No package re-export relation. No new resolver arm. A module is not a type, and the fix is to stop asking for one.
Three refusals keep it honest: a same-named declaration in the file, a
bindingrow (the name was rebound), and two imports binding one name.The graph half is fixed too, not just
find_callers— incomingsymbol_edgesondjango/__init__.py#setupwent 1 → 9.This issue is entirely about recall and never mentions the phantoms
The same mechanism was minting 106 wrong binds, and removing them is the larger half:
python-flask −16 / +0, every loss a phantom — 7 ×
flask.url_for → src/flask/app.py#url_for@1102(the Flask method; the module function ishelpers.py:200andflask/__init__.py:22isfrom .helpers import url_for as url_for), 4 ×click.command+ 3 ×click.group → src/flask/cli.py#FlaskGroup.*(there is nosrc/click— it is an external dependency), 2 ×_json.dump/load → provider.py(src/flask/json/__init__.py:3isimport json as _json, so_jsonIS the stdlib). I verified all sixteen at source myself before blessing the baseline.py-django −90 / +23. All 90 losses are one class:
copy.copy → forms/utils.py#copy,os.listdir → storage/base.py#listdir,json.dumps → core/signing.py#dumps@165from line 166 of the same file. Gains: 10 correct (9 ×django.setup() → django/__init__.py#setup@8) and 13 phantoms — stdlib module names colliding with in-project file stems, admitted by tier 1Q's stem anchor. That is #168's own family reached through a door this fix opened, named in the code with the closing shape (require the qualifier to be a prefix of the file's true module path), and the root gate cannot see it because a stdlib name is not a package tail.Reported rather than rounded away, as was py-django's
apps.readygain — a narrowing that turns an ambiguous stem pool into a unique wrong answer.The baseline moved and I blessed it, having read every row
corpus_ratchetwent red on one repo:python-flask resolved 3052 → 3036. The lane left it red rather than blessing, which is the standing rule. I adjudicated all 16 at source (above), confirmed six repos bind-for-bind identical against ade98a4fbinary, and blessed with that reason. The count fell because the count included phantoms.Your own worked example is still unresolved, for a third reason nobody named
flask.url_forinpython-flaskdoes not resolve after this fix — becausesrc/is aSRC_DIRShead, so every file undersrc/flask/has an emptypackage_root_ofand therefore an empty package tail, andflaskcan anchor on no package. That is #168'slib/hazard on the other repository. The class this issue describes is fixed; the headline repro needs that separate defect closed.Mutations
P1 (delete
requalify_module_calls) → RED ×3 (plugin, resolver, precision). P2 (alias binds the module root, not the dotted path) → RED. P3 (collectfrom-imports too) → RED: "from m import xbinds a VALUE as often as a module". P4/P5/P6 (drop each refusal) → RED.My first precision fixture let P1 survive — a two-file same-name pair, where the receiver-phantom gate refused the shape independently. Rebuilt as three sites reproducing the tiers the corpus phantoms actually used (rule 20 same-file,
src/flask/cli.py's shape verbatim, and rule 30 import boost); P1 is now RED on all three. That is a mutation surviving because a different gate caught it, caught inside the lane's own new work.shape_excluded_fallback_e2efailed correctly — this fix turned its python site from filter-hidden into honestly counted; the arm was rebuilt around a receiver call the filter still removes, with the improvement pinned (name_fallback_count: 1) so it cannot silently regress.precision_gatere-run by me: 7/7,phantom_count == 0,recall = 1.000, pythonforbid_sites7 → 8, probes 14 → 16.Closing.
package_root_ofreturns EMPTY when a SRC_DIRS name is the first path component, sosrc/- andlib/-headed trees can anchor no qualifier — #175's own repro still fails because of it #247correct_via_fallbackis gated by nothing #249django/utils/{inspect,html}.py) #251package_root_ofreturns EMPTY when a SRC_DIRS name is the first path component, sosrc/- andlib/-headed trees can anchor no qualifier — #175's own repro still fails because of it #247