h-dv/code-index

Watch

Fork 0

BLOCKER: no shipped path can grant `derived_names`, so a package using #86 gap 1 can never be enabled #103

New issue

Closed

opened 2026-09-04 13:53:33 +02:00 by buildagent · 1 comment

buildagent commented

2026-09-04 13:53:33 +02:00

Member

Found by #84 Phase 3 while porting Ruby to a package, and verified in source. This blocks #84's acceptance and therefore #75's final gate step 13.

The defect

#86 gap 1 shipped the derived-ref marker: a package may emit a ref whose NAME is derived rather than copied from the source, so its span means "where this fact came from" rather than "the bytes of this name". Ruby's Rails DSL is the motivating case — has_many :posts emits a type ref named Post at the span of the literal :posts.

Using it requires the derived_names grant. Nothing can give it.

HostPolicy::default() sets derived_names: false (crates/plugin-supervisor/src/supervise.rs:173).
PackageHost::discover and conform::run both build their policy with ..HostPolicy::default().
There is no CLI flag — grep derived_names crates/cli/src is empty.
There is no approval-record field — grep derived_names crates/indexer/src/approval.rs is empty.
Supervisor::set_derived_names exists but no shipped caller reaches it.

The one place that passes true says so itself, and says it is not a grant:

THE ONE PLACE IN ANY SHIPPED PATH THAT PASSES derived_names: true, and it is a PROBE rather than a grant: the ValidatedFacts it produces is dropped on the same expression that tests it, so no fact it admitted can reach a consumer. The refusal stands either way — this only decides what the refusal SAYS.

So the system can diagnose the missing grant precisely and cannot give it.

Why it is fatal rather than inconvenient

An ungranted resolver capability is INERT. An ungranted derived_names is FAIL-CLOSED.

That asymmetry is the whole problem. The capability model's default-deny is safe for a capability whose absence means "your symbols appear in search but join no pool". For derived_names, absence means validate refuses the whole file:

user.rb.rbx -> 0 symbols
files.parse_error = "package refused: fact.span_name_mismatch (this body validates with
                     the derived-name grant and not without it…)"

Measured directly: derived_names=false → fact.span_name_mismatch; derived_names=true → 2 symbols, 7 refs.

And through the shipped binaries, the chain stops:

plugin pack / digest / key generate / sign / trust add / install --sha256   OK
plugin check   -> core, members, visibility, broken: facts match
                  rails.rb.rbx: REFUSED fact.span_name_mismatch
plugin enable  -> refuses with conformance_not_run

So the acceptance package for #84 — the one that proves general language extensibility, which #75 says markup alone does not — cannot be enabled by an operator.

What the fix needs

An operator-facing way to answer yes, on the same footing as the other capability grants:

a field on the approval record, so the answer is durable and per-project;
a CLI surface to set it (plugin enable already takes capability grants — this belongs beside them);
PackageHost::discover and conform::run reading the grant instead of ..HostPolicy::default();
the grant reaching conform::run too, or plugin check keeps failing a package the operator has approved — C1's verdict is what enable reads, so a package that cannot pass conformance cannot be enabled regardless of the runtime grant.

The refusal witness is already excellent and should stay: it names the exact condition and does not guess. It just needs a reachable answer, which is the same defect class as plugin_add's poll instruction naming a state that could never become true.

Design question worth deciding explicitly

Should derived_names be inert-on-deny rather than fail-closed — i.e. drop the derived refs and keep the rest of the file, disclosing what was dropped? That would match how resolver behaves and would mean an ungranted package degrades instead of disappearing.

Arguments against: a package that asserts names it did not copy is exactly the thing the span check exists to police, and silently dropping facts is its own honesty problem. Arguments for: losing 100% of a file's symbols because one ref in it is derived is a very large blast radius for a capability the operator was never asked about.

Either answer is defensible; the current state — fail-closed with no way to grant — is not.

Blocks #84 (step 13 of #80's final gate) and therefore #75. Found alongside #102 (a real private_class_method defect in the builtin) by the same porting exercise.

Found by #84 Phase 3 while porting Ruby to a package, and verified in source. **This blocks #84's acceptance and therefore #75's final gate step 13.** ## The defect `#86 gap 1` shipped the derived-ref marker: a package may emit a ref whose NAME is derived rather than copied from the source, so its span means "where this fact came from" rather than "the bytes of this name". Ruby's Rails DSL is the motivating case — `has_many :posts` emits a `type` ref named `Post` at the span of the literal `:posts`. Using it requires the `derived_names` grant. **Nothing can give it.** - `HostPolicy::default()` sets `derived_names: false` (`crates/plugin-supervisor/src/supervise.rs:173`). - `PackageHost::discover` and `conform::run` both build their policy with `..HostPolicy::default()`. - There is **no CLI flag** — `grep derived_names crates/cli/src` is empty. - There is **no approval-record field** — `grep derived_names crates/indexer/src/approval.rs` is empty. - `Supervisor::set_derived_names` exists but no shipped caller reaches it. The one place that passes `true` says so itself, and says it is not a grant: > **THE ONE PLACE IN ANY SHIPPED PATH THAT PASSES `derived_names: true`, and it is a PROBE rather than a grant**: the `ValidatedFacts` it produces is dropped on the same expression that tests it, so no fact it admitted can reach a consumer. The refusal stands either way — this only decides what the refusal SAYS. So the system can *diagnose* the missing grant precisely and cannot *give* it. ## Why it is fatal rather than inconvenient **An ungranted `resolver` capability is INERT. An ungranted `derived_names` is FAIL-CLOSED.** That asymmetry is the whole problem. The capability model's default-deny is safe for a capability whose absence means "your symbols appear in search but join no pool". For `derived_names`, absence means `validate` **refuses the whole file**: ``` user.rb.rbx -> 0 symbols files.parse_error = "package refused: fact.span_name_mismatch (this body validates with the derived-name grant and not without it…)" ``` Measured directly: `derived_names=false` → `fact.span_name_mismatch`; `derived_names=true` → 2 symbols, 7 refs. And through the shipped binaries, the chain stops: ``` plugin pack / digest / key generate / sign / trust add / install --sha256 OK plugin check -> core, members, visibility, broken: facts match rails.rb.rbx: REFUSED fact.span_name_mismatch plugin enable -> refuses with conformance_not_run ``` So the acceptance package for #84 — the one that proves general language extensibility, which #75 says markup alone does not — **cannot be enabled by an operator**. ## What the fix needs An operator-facing way to answer yes, on the same footing as the other capability grants: 1. a field on the approval record, so the answer is durable and per-project; 2. a CLI surface to set it (`plugin enable` already takes capability grants — this belongs beside them); 3. `PackageHost::discover` and `conform::run` reading the grant instead of `..HostPolicy::default()`; 4. the grant reaching `conform::run` too, or `plugin check` keeps failing a package the operator has approved — C1's verdict is what `enable` reads, so a package that cannot pass conformance cannot be enabled regardless of the runtime grant. The refusal witness is already excellent and should stay: it names the exact condition and does not guess. It just needs a reachable answer, which is the same defect class as `plugin_add`'s poll instruction naming a state that could never become true. ## Design question worth deciding explicitly Should `derived_names` be **inert-on-deny** rather than fail-closed — i.e. drop the derived refs and keep the rest of the file, disclosing what was dropped? That would match how `resolver` behaves and would mean an ungranted package degrades instead of disappearing. Arguments against: a package that asserts names it did not copy is exactly the thing the span check exists to police, and silently dropping facts is its own honesty problem. Arguments for: losing 100% of a file's symbols because one ref in it is derived is a very large blast radius for a capability the operator was never asked about. Either answer is defensible; the current state — fail-closed with no way to grant — is not. ## Related Blocks #84 (step 13 of #80's final gate) and therefore #75. Found alongside #102 (a real `private_class_method` defect in the builtin) by the same porting exercise.

buildagent commented

2026-09-04 15:00:52 +02:00

Author

Member

Fixed. The grant is reachable, durable, per-project, and re-validated at read time.

The path, end to end

Approval record — Approval.derived_names (#[serde(default)], absent reads as deny) plus a #[must_use] with_derived_names(bool) builder rather than a 5th parameter to Approval::granted, so all 16 existing call sites keep the honest default. StoredPackage.requested_derived_names parses the manifest's ask and is the ceiling.
verify_grant rule 5, checked first: derived_names && !entry.requested_derived_names → capability_not_granted. First because a package holding this can bind any name at any in-range span, so its refusal must not queue behind a typo in an unrelated capability list.
CLI — plugin enable <digest> --derived-names and plugin check <digest> --derived-names, beside --capabilities/--bridges as a flag rather than a name inside them (it is not a resolver capability and RESOLVER_CAPABILITIES is a closed registry). --grant requested answers the manifest's whole request. plugin add needed no flag — it resolves the grant before C1 and threads one answer into both.
Runtime — PackageHost::start replaces HostPolicy::default() with the grant read from the set. derived_names_granted() is a three-way AND: project ceiling && manifest request && this package's grant. The third conjunct is load-bearing: one Supervisor serves a whole lane pool, so without it the first package an operator granted would arm every other package that merely asked.
effective_grant now calls verify_grant, not validate_grant — a hand-edited derived_names = true for a package that never asked empties the grant rather than conferring it.

The conformance chicken-and-egg, resolved explicitly

C1 runs before approval by design, and enable refuses a package with no passing verdict. For every other capability that is free, because an ungranted resolver capability is inert. derived_names is fail-closed, so a package whose fixtures contain a derived name cannot pass C1 at all and could never be enabled whatever the operator would have granted.

Resolution: the answer is an argument to both commands, and the verdict records which answer the run held. Verdict.derived_names, #[serde(default)], and no VERDICT_SCHEMA bump — bumping would silently un-enable every already-checked package, because Verdict::load returns None for an unknown schema and None is conformance_not_run. New require_verdict_covering_grant adds one clause: a verdict earned without the authority cannot license a grant that has it. The reverse direction is deliberately allowed — refusing it would deny an operator the right to say no.

Both directions, mutation-proved (all RUN, all RED)

mutation	result
delete rule 5	RED — `a package that did not ask must not be granted derived names`
`plan_grant` `Requested` → literal `false`	RED — `requested` must answer the manifest's whole request
`plan_grant` `Requested` → literal `true`	RED — refused by rule 5, witness quoted
`GrantSpec::named` → `true`	RED — `the low-level form defaults to DENY`
drop `#[serde(default)]`	RED — `a record written before #103 must still load`
drop `&& self.derived_names_grant`	RED — `a project ceiling must not arm a package this project did not grant`
`require_verdict_covering_grant` guard neutered	RED
also refuse the reverse direction	RED — `an operator must stay free to say no`
`PackageHost::start` back to `HostPolicy::default()` — this defect, reintroduced	RED at the e2e — the file carries the `fact.span_name_mismatch` refusal again
`check_with_grant` records `derived_names: false`	RED — the CLI prints that line off the verdict, not off the flag
`set_derived_names(true)` unconditionally	RED — the ungranted witness stops being recorded

One mutation SURVIVED and is recorded rather than hidden: the check_with_grant mutation is green against the unit test, which builds its two verdicts by hand and never runs the writer. That measurement is written into the test's own doc, pointing at the e2e that does catch it.

Acceptance through the shipped binaries

an_operator_can_grant_derived_names_and_the_rails_file_then_indexes drives the real code-index binary — deliberately through a Cli harness rather than the internal fx::Operator fixture, because this was a defect in the operator surface and a fixture reaching past it could neither catch it nor prove it fixed. pack → sign → trust add → install --sha256 → check (fails, rails.rb.rbx REFUSED) → check --derived-names (all five fixtures extract, verdict passes) → enable --derived-names → real daemon: no parse_error, class User indexes, and the derived ref type Post — four bytes that appear nowhere in the file — is present and returned by search_symbols. Both ungranted witnesses still pass unchanged.

Pre-#103 approval records still load, graded against a hand-written TOML rather than this build's serializer agreeing with itself. RECORD_SCHEMA and VERDICT_SCHEMA both unmoved.

The design question: keep fail-closed. Do not implement inert-on-deny.

Its premise is gone. The argument for it was "a capability the operator was never asked about". The operator is now asked twice, with a disclosure stating the cost in both directions.
Dropping a derived ref is not the same fact as withholding a pool. An ungranted resolver capability leaves the symbol rows identical and only narrows resolution — and resolution_gaps / name_fallback_count already tell a reader what was not resolved. Dropping a derived ref changes the facts with no per-row channel to say so: find_callers on Post would return a truthful-looking zero. That is the absent-reads-as-negative shape this project refuses everywhere.
The honesty channel does not exist and would cost more than this fix did. Today files.parse_error carries the whole story and it is measured — withheld_grant_witness re-validates the same body with the grant and observes it pass. Inert-on-deny leaves no error, so it would need a new per-file "n refs dropped" column, a reader surface, and a rule binding every ref-reporting tool to disclose it.
It would make C1 worse. An ungranted check would produce fewer facts and fail the expectation comparison anyway — so the operator still needs the flag, but the failure arrives as a fact-mismatch instead of a named refusal with a repair.
The measured blast radius is per-FILE, not per-package. 1 of 5 fixtures and 1 of 2 corpus files; greeter.rb.rbx indexes fully with no grant. "The package disappears" is not what happens.

Gates: fmt, clippy (native and x86_64-pc-windows-gnu), rustdoc, cargo test --workspace (239 binaries), daemon-leg e2e, precision_gate 7/7 — all green. tests/corpus/baseline.json unmoved at 534084b856c22566c48e386bc41ed67e.

Follow-ups filed separately rather than scope-crept into this fix: the re-grant reindex gap, and making the running cost of declining visible in aggregate.

## Fixed. The grant is reachable, durable, per-project, and re-validated at read time. ### The path, end to end 1. **Approval record** — `Approval.derived_names` (`#[serde(default)]`, absent reads as **deny**) plus a `#[must_use] with_derived_names(bool)` builder rather than a 5th parameter to `Approval::granted`, so all 16 existing call sites keep the honest default. `StoredPackage.requested_derived_names` parses the manifest's ask and is the **ceiling**. 2. **`verify_grant` rule 5**, checked **first**: `derived_names && !entry.requested_derived_names` → `capability_not_granted`. First because a package holding this can bind any name at any in-range span, so its refusal must not queue behind a typo in an unrelated capability list. 3. **CLI** — `plugin enable <digest> --derived-names` and `plugin check <digest> --derived-names`, beside `--capabilities`/`--bridges` as a flag rather than a name inside them (it is not a resolver capability and `RESOLVER_CAPABILITIES` is a closed registry). `--grant requested` answers the manifest's *whole* request. `plugin add` needed no flag — it resolves the grant before C1 and threads one answer into both. 4. **Runtime** — `PackageHost::start` replaces `HostPolicy::default()` with the grant read from the set. `derived_names_granted()` is a **three-way AND**: project ceiling && manifest request && *this package's* grant. The third conjunct is load-bearing: one `Supervisor` serves a whole lane pool, so without it the first package an operator granted would arm every other package that merely asked. 5. **`effective_grant` now calls `verify_grant`, not `validate_grant`** — a hand-edited `derived_names = true` for a package that never asked empties the grant rather than conferring it. ### The conformance chicken-and-egg, resolved explicitly C1 runs *before* approval by design, and `enable` refuses a package with no passing verdict. For every other capability that is free, because an ungranted resolver capability is inert. `derived_names` is fail-closed, so a package whose fixtures contain a derived name **cannot pass C1 at all** and could never be enabled whatever the operator would have granted. Resolution: **the answer is an argument to both commands, and the verdict records which answer the run held.** `Verdict.derived_names`, `#[serde(default)]`, and **no `VERDICT_SCHEMA` bump** — bumping would silently un-enable every already-checked package, because `Verdict::load` returns `None` for an unknown schema and `None` is `conformance_not_run`. New `require_verdict_covering_grant` adds one clause: a verdict earned **without** the authority cannot license a grant that **has** it. The reverse direction is deliberately allowed — refusing it would deny an operator the right to say no. ### Both directions, mutation-proved (all RUN, all RED) | mutation | result | |---|---| | delete rule 5 | RED — `a package that did not ask must not be granted derived names` | | `plan_grant` `Requested` → literal `false` | RED — `` `requested` must answer the manifest's whole request `` | | `plan_grant` `Requested` → literal `true` | RED — refused by rule 5, witness quoted | | `GrantSpec::named` → `true` | RED — `the low-level form defaults to DENY` | | drop `#[serde(default)]` | RED — `a record written before #103 must still load` | | drop `&& self.derived_names_grant` | RED — `a project ceiling must not arm a package this project did not grant` | | `require_verdict_covering_grant` guard neutered | RED | | also refuse the reverse direction | RED — `an operator must stay free to say no` | | **`PackageHost::start` back to `HostPolicy::default()`** — this defect, reintroduced | RED at the e2e — the file carries the `fact.span_name_mismatch` refusal again | | `check_with_grant` records `derived_names: false` | RED — the CLI prints that line **off the verdict**, not off the flag | | `set_derived_names(true)` unconditionally | RED — the ungranted witness stops being recorded | **One mutation SURVIVED and is recorded rather than hidden:** the `check_with_grant` mutation is green against the *unit* test, which builds its two verdicts by hand and never runs the writer. That measurement is written into the test's own doc, pointing at the e2e that does catch it. ### Acceptance through the shipped binaries `an_operator_can_grant_derived_names_and_the_rails_file_then_indexes` drives the real `code-index` binary — deliberately through a `Cli` harness rather than the internal `fx::Operator` fixture, because this was a defect in the **operator surface** and a fixture reaching past it could neither catch it nor prove it fixed. pack → sign → trust add → install --sha256 → `check` (fails, `rails.rb.rbx REFUSED`) → `check --derived-names` (all five fixtures extract, verdict passes) → `enable --derived-names` → real daemon: no `parse_error`, `class User` indexes, and the derived ref `type Post` — four bytes that appear nowhere in the file — is present and returned by `search_symbols`. Both ungranted witnesses still pass unchanged. Pre-#103 approval records still load, graded against a **hand-written** TOML rather than this build's serializer agreeing with itself. `RECORD_SCHEMA` and `VERDICT_SCHEMA` both unmoved. ## The design question: **keep fail-closed. Do not implement inert-on-deny.** 1. **Its premise is gone.** The argument for it was "a capability the operator was never asked about". The operator is now asked twice, with a disclosure stating the cost in both directions. 2. **Dropping a derived ref is not the same fact as withholding a pool.** An ungranted resolver capability leaves the symbol rows identical and only narrows resolution — and `resolution_gaps` / `name_fallback_count` already tell a reader what was not resolved. Dropping a derived ref changes the **facts** with no per-row channel to say so: `find_callers` on `Post` would return a truthful-looking zero. That is the absent-reads-as-negative shape this project refuses everywhere. 3. **The honesty channel does not exist and would cost more than this fix did.** Today `files.parse_error` carries the whole story and it is **measured** — `withheld_grant_witness` re-validates the same body with the grant and observes it pass. Inert-on-deny leaves no error, so it would need a new per-file "n refs dropped" column, a reader surface, and a rule binding every ref-reporting tool to disclose it. 4. **It would make C1 worse.** An ungranted check would produce *fewer* facts and fail the expectation comparison anyway — so the operator still needs the flag, but the failure arrives as a fact-mismatch instead of a named refusal with a repair. 5. **The measured blast radius is per-FILE, not per-package.** 1 of 5 fixtures and 1 of 2 corpus files; `greeter.rb.rbx` indexes fully with no grant. "The package disappears" is not what happens. Gates: fmt, clippy (native **and** `x86_64-pc-windows-gnu`), rustdoc, `cargo test --workspace` (239 binaries), daemon-leg e2e, `precision_gate` 7/7 — all green. `tests/corpus/baseline.json` unmoved at `534084b856c22566c48e386bc41ed67e`. Follow-ups filed separately rather than scope-crept into this fix: the re-grant reindex gap, and making the running cost of declining visible in aggregate.

buildagent referenced this issue

2026-09-04 15:01:11 +02:00

Re-granting derived_names on an already-enabled package does not reindex, so the files stay at 0 symbols #104

buildagent referenced this issue

2026-09-04 15:01:21 +02:00

The running cost of declining derived_names is only visible by grepping files.parse_error #105

buildagent referenced this issue

2026-09-04 15:01:35 +02:00

The MCP plugin_add tool cannot grant derived_names, so an agent can never add a package that needs it #106

buildagent closed this issue

2026-09-04 15:01:37 +02:00

buildagent referenced this issue