Nothing grades that a `plugin` subcommand is documented — `plugin update` shipped with ZERO operator documentation and every gate green #256

New issue

Closed

opened 2026-09-10 16:12:51 +02:00 by buildagent · 0 comments

buildagent commented

2026-09-10 16:12:51 +02:00

Member

The measurement

At ee39fd1, with plugin update (#237) fully shipped and printing its own name to operators, these were the counts across every .md and text file in the tree (search_text with category="text", each negative carrying empty_population.basis: "filtered" naming the filter, and each spelling measured separately via separator_scan):

term	hits in text	hits in code
`auto_apply`	0	6
`auto_update`	0	0
`[update]` / `[update."`	0	4
`plugin update`	0	6
`update_check`	0	3
`update_publisher_changed`	0	2

Positive control: install over the same path_glob returns all 10 guides, 67 hits in the recovery guide alone. The zeros are measurements, not vacuous queries.

Meanwhile crates/cli/src/plugin_doctor.rs:1113 prints, to operators, a string naming a command no shipped document contained:

NOT MEASURED — no code-index plugin update has run for this project.

and :1123:

re-run code-index plugin update --check to rewrite it

An operator told to run a command could not look it up. _prdoc/guides/80-operator-recovery.md §10 — the section whose whole job is "the rest of the command surface, so a reader does not go looking" — enumerated every other plugin subcommand and omitted update, while §9 stated "There is no plugin upgrade, and this is a real gap" next to a shipped cmd_update.

Why no gate saw it

Two gates could have and neither is scoped to it:

crates/mcp-server/tests/shipped_binary_registry.rs::the_readme_cli_reference_names_every_shipped_verb (:463-490) — readme_cli_verbs (:333-342) takes only "the word after code-index at the start of a line", i.e. top-level verbs. plugin is present on both sides, so the entire subcommand family is invisible to it. plugin update, plugin gc, plugin doctor, plugin trust remove are all equally invisible.
crates/cli/tests/plugin_command_registry.rs (1088 lines, 6 tests) — enumerates every PluginCommand variant and asserts set-equality with its own REGISTRY, then grades disclosure (the four required payload fields, the abi_stability line, the mutating/read-only split). Update is row :166. Its module doc lists what it cannot catch, and documentation was never in scope. Its one doc-touching test (the_abi_is_marked_experimental_on_every_static_surface, :1005-1023) checks 4 tokens on 3 surfaces, none of them README or the guides.

So the registry that already knows the complete, authoritative list of subcommands does not ask whether any of them is documented. A plugin subcommand can ship, print its own name to an operator, and appear in no document — which is what happened.

Suggested shape (generic, not a fix per case)

plugin_command_registry.rs already holds the ground truth and already asserts set-equality against the enum, with anti-vacuity (declared.len() >= 20). Extend that one registry with a documentation axis rather than adding a per-command doc test:

for each row, assert its invocation (code-index plugin <verb>) appears in at least one operator-facing document — README.md plus _prdoc/guides/*.md is the set the doc_citation_gate already walks;
anti-vacuity in both directions, the way test 2 in that file already does it: a verb documented but not in the enum is also a failure (that is how §9's stale plugin upgrade claim would have been caught);
the failure message should name the verb and the documents searched, so the fix is obvious.

The honest limit to write into the test: matching a string is not "documented well." It cannot see a wrong explanation, only a missing one. That is still the difference between this finding and no finding.

_prdoc/guides/80-documentation-index.md exists so #80's Documentation checklist "can be checked mechanically rather than from memory" (:3-4), but is scoped to #80's ten items and nothing adds itself. #237 had no row, no owed-item note, and no "not covered here" mention, so the index read green while the feature was undocumented. (Lane C added a "Documentation this table is NOT scoped to" section and a #237 row; the structural point stands — nothing in CI grades that table either.)

What Lane C already fixed, so this issue is about the gate only

README.md — plugin update in the command synopsis plus an Updating a package section (the [update] block, the six reason codes, and the fact that nothing schedules it).
_prdoc/guides/80-operator-recovery.md §9 — rewritten: §9.1 plugin update, §9.2 plugin add, §9.3 the four commands; the false "there is no upgrade path, it takes four commands" claim removed; §10's table row added.
_prdoc/guides/80-documentation-index.md — the out-of-scope section above.

None of that is enforced. Without the gate the next subcommand repeats it.

## The measurement At `ee39fd1`, with `plugin update` (#237) fully shipped and printing its own name to operators, these were the counts across **every `.md` and text file in the tree** (`search_text` with `category="text"`, each negative carrying `empty_population.basis: "filtered"` naming the filter, and each spelling measured separately via `separator_scan`): | term | hits in text | hits in code | |---|---|---| | `auto_apply` | **0** | 6 | | `auto_update` | **0** | 0 | | `[update]` / `[update."` | **0** | 4 | | `plugin update` | **0** | 6 | | `update_check` | **0** | 3 | | `update_publisher_changed` | **0** | 2 | Positive control: `install` over the same `path_glob` returns all 10 guides, 67 hits in the recovery guide alone. The zeros are measurements, not vacuous queries. Meanwhile `crates/cli/src/plugin_doctor.rs:1113` prints, to operators, a string naming a command no shipped document contained: > `NOT MEASURED — no `code-index plugin update` has run for this project.` and `:1123`: > `re-run `code-index plugin update --check` to rewrite it` An operator told to run a command could not look it up. `_prdoc/guides/80-operator-recovery.md` §10 — the section whose whole job is "the rest of the command surface, so a reader does not go looking" — enumerated every other `plugin` subcommand and omitted `update`, while §9 stated *"There is no `plugin upgrade`, and this is a real gap"* next to a shipped `cmd_update`. ## Why no gate saw it Two gates could have and neither is scoped to it: * **`crates/mcp-server/tests/shipped_binary_registry.rs::the_readme_cli_reference_names_every_shipped_verb`** (`:463-490`) — `readme_cli_verbs` (`:333-342`) takes only *"the word after `code-index ` at the start of a line"*, i.e. **top-level verbs**. `plugin` is present on both sides, so the entire subcommand family is invisible to it. `plugin update`, `plugin gc`, `plugin doctor`, `plugin trust remove` are all equally invisible. * **`crates/cli/tests/plugin_command_registry.rs`** (1088 lines, 6 tests) — enumerates every `PluginCommand` variant and asserts set-equality with its own `REGISTRY`, then grades **disclosure** (the four required payload fields, the `abi_stability` line, the mutating/read-only split). `Update` is row `:166`. Its module doc lists what it cannot catch, and documentation was never in scope. Its one doc-touching test (`the_abi_is_marked_experimental_on_every_static_surface`, `:1005-1023`) checks 4 tokens on 3 surfaces, none of them README or the guides. So the registry that already knows the complete, authoritative list of subcommands does not ask whether any of them is documented. **A `plugin` subcommand can ship, print its own name to an operator, and appear in no document — which is what happened.** ## Suggested shape (generic, not a fix per case) `plugin_command_registry.rs` already holds the ground truth and already asserts set-equality against the enum, with anti-vacuity (`declared.len() >= 20`). Extend that one registry with a documentation axis rather than adding a per-command doc test: * for each row, assert its **invocation** (`code-index plugin <verb>`) appears in at least one operator-facing document — `README.md` plus `_prdoc/guides/*.md` is the set the `doc_citation_gate` already walks; * anti-vacuity in both directions, the way test 2 in that file already does it: a verb documented but not in the enum is also a failure (that is how §9's stale `plugin upgrade` claim would have been caught); * the failure message should name the verb and the documents searched, so the fix is obvious. The honest limit to write into the test: **matching a string is not "documented well."** It cannot see a wrong explanation, only a missing one. That is still the difference between this finding and no finding. ## Related, same root `_prdoc/guides/80-documentation-index.md` exists so #80's Documentation checklist "can be checked mechanically rather than from memory" (`:3-4`), but is scoped to #80's ten items and nothing adds itself. #237 had no row, no owed-item note, and no "not covered here" mention, so the index read green while the feature was undocumented. (Lane C added a "Documentation this table is NOT scoped to" section and a #237 row; the structural point stands — nothing in CI grades that table either.) ## What Lane C already fixed, so this issue is about the gate only * `README.md` — `plugin update` in the command synopsis plus an *Updating a package* section (the `[update]` block, the six reason codes, and the fact that nothing schedules it). * `_prdoc/guides/80-operator-recovery.md` §9 — rewritten: §9.1 `plugin update`, §9.2 `plugin add`, §9.3 the four commands; the false "there is no upgrade path, it takes four commands" claim removed; §10's table row added. * `_prdoc/guides/80-documentation-index.md` — the out-of-scope section above. None of that is enforced. Without the gate the next subcommand repeats it.