plugin: a package cannot replace a builtin, and a guest cannot name a node kind #87
Labels
No labels
code-review
correctness
dos
performance
security
severity/high
severity/low
severity/medium
tech-debt
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
h-dv/code-index#87
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Two structural blockers between "packages add languages beside the builtins" and the stated direction, "the base system builds only infrastructure; all languages are plugins". Measured during #80's pre-release review.
Neither is an ABI gap (those are #86). Both are about what a package is allowed and able to be.
1. A package may not claim a file a builtin claims, and there is no override
Manifest::validateruns a claim check against a compile-timestatic BUILTIN_CLAIMSand refuses any package that overlaps it. There is no per-project override.languages.enabledlooks like the escape hatch and is not:crates/indexer/src/activation.rs's own doc says it is "read by NOTHING in this workspace outside its own tests".So today a package can only ever be additive. Under the stated direction the builtin set eventually shrinks to zero, and every step of that path requires a package to take over a claim a builtin currently holds.
Why it is not a one-line change. Making the effective builtin set project-scoped makes it an activation-identity input — two projects on one daemon would disagree about who owns
.rb, and the activation digest has to say so. It probably needs a migration. It is a product decision about who may disable a builtin and how that is disclosed, not a config toggle.Workaround that exists today, and its limit. #84's Ruby migration uses a shadow extension, which gets the parity gate green without touching this. That proves the extraction path; it does not prove replacement.
2. A guest cannot map a kind NAME to a kind ID
The tree stream a guest walks carries
u16kind ids andu16field ids (crates/plugin-host/src/tree.rs). The guest is instantiated with an empty import list — nots_language_symbol_for_name, no string table, no host call of any kind. That zero-import property is load-bearing for containment (it is what makesfork/open/connectinexpressible) and should not be traded away.So a guest cannot ask "what number is
function_item?" It can only compare numbers.The XAML reference package sidesteps this by keying on SOURCE TEXT. That works for markup, where the interesting nodes are attributes with distinctive spellings. It cannot work for a language: nothing in a
function_item's bytes distinguishes it from acall_expression's.What is needed: a kind/field id table generated from the grammar artifact the package ships, compiled into the guest — plus a test proving the wasm grammar and the native crate agree on every id, not just the ones one plugin happens to use. If those two disagree, every fact a guest emits is silently wrong and the parity gate goes red with no diagnosable cause.
Note this is per-artifact, not per-language: it must be regenerated whenever the grammar is rebuilt, which is why it belongs beside the reproducible grammar build (
tests/grammars/build-tree-sitter-xml.sh) rather than in the SDK.Also required, and currently absent
_prdoc/guides/80-extractor-abi.md§5 states it plainly: nocode-index-sdkcrate, no guest-side helper library, no published example package. The only guests anywhere in the repository are hand-written.watfixtures..wat. Whether awasm32-unknown-unknowncdylib satisfies it, exports the required symbols, precompiles, and lands underMAX_EXTRACTOR_BYTESis unknown — and it is the step that can end #84 early, which is why #84's plan puts it before any SDK work.Related
#84 (first migration — uses a shadow extension to avoid blocker 1), #85 (extraction serialization — the throughput blocker), #86 (the three ABI wire gaps), #80. Direction recorded in
_prdoc/records/80-S43-migration-candidate.md.Half shipped in v0.26.0 — the kind-naming half. The builtin-replacement half is untouched.
"A guest cannot name a node kind" — CLOSED.
A guest walks a tree of
u16kind ids with an empty import list, and that emptiness is what makesfork,openandconnectinexpressible — so it cannot ask the host "what number isfunction_item?" without giving up the property that makes the sandbox worth having. The XAML package sidesteps this by keying on source text, which works for markup and cannot work for a language: nothing in afunction_item's bytes distinguishes it from acall_expression's.So the guest does not ask — it asserts and the host checks. The guest exports one immutable
kind_table_digest; the worker enumerates the grammar it has just loaded, in the same process whosetree::serializewrites those ids, and refuses on mismatch with its own reason code and exit 24, printing the number the guest should have carried.Two properties are measured rather than asserted:
Module::imports()rather than claimed in prose.node_kind_is_visiblewherenode_kind_is_namedbelongs SURVIVED the wasm-vs-native comparison; only the Node-API oracle on a real parse kills it. A same-implementation parity test would have shipped that bug."A package cannot replace a builtin" — still open, nothing done.
That half remains as filed. It is the harder of the two: it is about precedence and ownership in the claim algebra, not about handing a guest a lookup table, and it interacts with #78's generation model and with the duplicate-id ordering settled in #91.
Retitling or splitting this issue would make the remaining scope clearer, but I have not done so unilaterally.
GUEST_ABI_MAJORis bracketed by no manifest field, andplugin pack --check-reproducibledoes not exist #153Decision made by the owner, 2026-09-05
A package MAY claim an extension a builtin owns — but only with explicit operator consent at
plugin addtime, and the displacement must be shown in the confirmation the operator answers.Silence keeps today's refusal. The default does not change. Implementation is in flight.
What forced the decision
A pluggability review measured the current state and it is starker than this issue's title suggests:
Refused at pack time, before the ABI is consulted.
BUILTIN_CLAIMSincrates/package/src/builtin.rsis checked unconditionally in bothcrates/package/src/manifest.rsandcrates/indexer/src/packages.rs, with no override and no knob.html,vue,go,java,kt,sql,erball pack fine.So zero of six builtins could ship as a package, and the reason sits upstream of every ABI gap in #86.
The consequence worth stating plainly: the Ruby package is not a Ruby package. It claims
.rbx, a shadow extension no repository has. It is a correctness proof running on a file type nobody uses. That is an honest deferral —tests/packages/README.mdsays so — but it means the anti-vacuity claim from #84 needs restating: it proves the architecture carries a real language; it does not prove the product can replace a builtin.Scope of the in-flight work
pack/validateaccept it; the package is marked as displacing.plugin add's confirmation (shared by CLI and the MCPplugin_addtool) shows which builtin loses which extensions, beside the grant and reindex domain it already shows. An operator must not be able to displace a builtin without seeing it.Explicitly not in scope: retiring the builtin Ruby extractor or deleting
tests/packages/ruby. That is a separate decision once displacement works.The second half of this issue's title — "a guest cannot name a node kind" — is confirmed and now written up in detail in #153, including the measurement that
tree-sitter-rubyspellscallwith four distinct ids, so a guest comparing one id silently misses three quarters of Ruby's call sites.🤖 Generated with Claude Code
https://claude.ai/code/session_01K1zj5VcFJvJt3pQxe9259K
Status of the second half, and a note the guide gap in #153 makes urgent
Nothing new landed here this round — the builtin-replacement half is the owner's in-flight
[[displaces]]work, and the kind-naming half closed in v0.26.0. Recording two things a pluggability re-measurement turned up that bear directly on this issue's text.1.
gen_kinds.pyis the answer to this issue's blocker 2, and it is documented nowhereThis issue asked for exactly it:
That exists.
crates/guest/ruby/gen_kinds.pyis 167 lines, it is genuinely reusable, andcrates/guest/ruby/gen-kinds.shrecords the measurement that makes it non-optional:tree-sitter-rubyspellscallwith FOUR distinct ids andassignmentwith two. A builtin comparingnode.kind()strings folds them for free; a guest comparing one id silently misses three quarters of Ruby's call sites, with no error and no diagnosable cause — the exact failure this issue predicted.It is mentioned in no guide.
_prdoc/guides/80-package-authoring.mddoes not name it, and #153 measured a third-party author getting from nothing to indexed rows without ever meeting it — which works for XAML (markup, keyed on source text) and cannot work for a language, for the reason this issue's own §2 gives.That is the single cheapest thing standing between "a guest CAN name a node kind" and "a third party can". The mechanism is done and proven; the instruction is missing. It belongs in the authoring guide beside the grammar build, per this issue's own note that it is per-artifact rather than per-language.
2. The workaround this issue names has a new consequence, now that #112 is fixed
Still exactly right, and there is now a second cost to it. After #112, the
ruby_package_paritydeltas are:SelfQualifier(#86, the fact ABI has no qualifier TEXT)EcosystemMarker— the shadow extension itselfEcosystemMarkeris four refs that resolve on the package leg and not on the builtin one, becauseindex.rs::manifest_package_dirsfinds a project's package roots by FILENAME (Gemfile,*.gemspec) and the shadow renames both, sotemp.file_pkg'spkg_dirdiffers and the origin gate on the file-key import arm moves with it.It is small, it is pinned by site, and it is honestly labelled a harness artifact rather than a defect. But it is now one of only two things left in a comparison whose whole purpose is to prove the packaged path is not second-class — and it exists solely because a package may not claim
.rb. When the operator-consented displacement lands, that pin should go to zero along with the workaround, and if it does not, that is a finding about displacement rather than about the harness.Worth stating because the arithmetic has changed: with #112 closed, the remaining gap between a packaged Ruby and the builtin one is one ABI gap (#86) and one consequence of this issue. Nothing else.
🤖 Generated with Claude Code
https://claude.ai/code/session_01K1zj5VcFJvJt3pQxe9259K
Triage 2026-09-06: CLOSING. Both halves shipped — and this issue's most recent comment is stale on half 1, which still calls the displacement work "in-flight".
Verified against master.
Half 2 — a guest can name a node kind (already acknowledged closed)
kind_table_digest—crates/plugin-host/src/globals.rs:7; generatorcrates/guest/ruby/gen_kinds.py:44emittingpub const KIND_TABLE_DIGEST; guest exportcrates/guest/ruby/src/lib.rs:224; host checkcrates/plugin-host/src/worker.rs:228-309.incl.
the_mechanism_adds_no_import,a_guest_compiled_against_another_grammar_is_refused,the_worker_process_exits_with_the_kind_table_status. The first of those is the one that matters for the ABI: the mechanism costs no import, so it does not weaken the zero-imports property the threat model rests on.Half 1 — a package may replace a builtin
Landed in
b668305(an ancestor of HEAD), and it shipped with operator consent as a first-class part of the design, not as a flag:crates/package/src/manifest.rs:142pub displaces, validated at:735-775; refusal message atcrates/package/src/builtin.rs:338.plugin addconfirmation:crates/indexer/src/confirm.rs:322pub displaces: Displacement, with the MCP copy asserted atcrates/mcp-server/tests/plugin_add_mcp_e2e.rs:1201("displaces_builtin (n…").crates/indexer/src/packages.rs:454pub displaces: Displaced;crates/indexer/src/approval.rs:2148; dirty/reindex domaincrates/indexer/src/dirty.rs:254-601.crates/package/src/digest.rs:198-299derives[[displaces]]into the digest, graded bytaking_a_builtin_extension_moves_the_extraction_identity. Displacement that did not move identity would be the dangerous version of this feature._prdoc/guides/80-package-authoring.md:479-573; refusal code incrates/mcp-server/src/docs/refusal-codes.md:116.The names carry the safety argument:
silence_refuses_exactly_as_before,consenting_to_one_key_does_not_consent_to_its_siblings,the_rule_is_one_clause_over_all_seven_builtins, and end-to-enda_consented_package_takes_rs_from_the_builtin_and_the_old_rows_stop_being_servedplusthe_silent_variant_never_reaches_the_store. Further coverage exists that I did not run:crates/indexer/tests/builtin_displacement_routing.rs,crates/cli/tests/plugin_displacement_cli.rs.The issue's "Also required, and currently absent" section is stale in two of three ways
crates/guest/ruby,extractor.wasm24,218 B.crates/guest/src/encode.rsis a guest-side helper library — it exists.code-index-sdk. That belongs to #153, which stays open and now carries a corrected title question.Carried forward rather than closed here
gen_kinds.py— the answer to blocker 2 — is named in no authoring guide. That is #153's territory.EcosystemMarker4-row parity pin should go to zero once the Ruby package actually claims.rb. If it does not, that is a displacement finding and should come back as a new issue.🤖 Triage lane, 2026-09-06, master
45cf6e4GUEST_ABI_MAJORis bracketed by no manifest field, andplugin pack --check-reproducibledoes not exist #153