h-dv/code-index

Watch

Fork 0

`evidence_gaps` warns on every reply that the answer may be short, and gives no way to find out which file — `index_coverage(path)` needs the path you are trying to learn #241

New issue

Closed

opened 2026-09-09 13:57:02 +02:00 by buildagent · 1 comment

buildagent commented

2026-09-09 13:57:02 +02:00

Member

Found by the #184/#205 lane, and independently confirmed by me across an entire working session — which is itself the evidence for the claim.

The fact

Every reply from this server on this repository carries:

"evidence_gaps": {
  "partial_sources": [],
  "partial_sources_in_index": 1,
  "semantics": "PARTIAL SOURCES: 1 file(s) … were produced by an extractor that
    reported an INCOMPLETE extraction … a count is a LOWER BOUND and an absent
    row is not evidence of absence … `index_coverage(path)` reports what each
    producer said."
}

The three-state discipline here is correct and I am not disputing it. partial_sources names the partial files this reply's own body names; partial_sources_in_index is a COUNT(DISTINCT path) over the whole index. So when the partial file is not among your results, you get 1 and [], and the design's own doc calls it exactly right:

it landed in partial_sources_in_index as an anonymous denominator and never in partial_sources as the named row

The defect

The remedy the disclosure names requires the answer it is withholding. index_coverage(path) reports what a producer said about one path you already know. The reader's question here is which path, and no surface answers it.

The options available to a caller are: call index_coverage on every file in the repository, or ignore the warning. Both are the wrong answer for a disclosure that appears on every single reply.

Why this is worse than a missing feature

A warning that cannot be acted on is trained away. I read partial_sources_in_index: 1 several dozen times in one session — while closing issues about disclosure honesty — and never once investigated it, because there was no next step. That is the failure mode: the disclosure is correct, permanent, and therefore invisible. It has become the wallpaper that compose_evidence_semantics's own doc comment warns against:

A constant paragraph describing states this reply is not in is how a disclosure becomes wallpaper.

The clause is not constant — it fires on a real condition — but from the reader's side it is indistinguishable from constant, because the condition never changes and nothing can be done about it.

What must NOT be done

Do not suppress it when partial_sources is empty. That would collapse "one file in this index is partial and it is not in your results" into "nothing to report" — losing a true fact, and re-creating #216 exactly.
Do not inline the whole list into every reply. On a large repo the partial set could be long, and this pays a per-reply cost for a fact most callers do not need. #197's payload budget rules that out.
Do not answer it with prose in the tool description. Description prose is unread and expensive; this repo has measured that.
Do not drop partial_sources' reply-scoped semantics. The named-row behaviour is right — a partial file among your results deserves to be named inline. This issue is only about the anonymous remainder.

Shape of a fix

The cheapest honest thing is a surface that enumerates them on demand, so the per-reply clause can point at a call that works:

a resource (code-index://index/partial-sources), or
a field on index_health / project_overview, which already carry index-wide facts and are called once rather than per-query, or
an index_coverage mode that takes no path and returns the partial set.

Then the semantics string's last sentence changes from index_coverage(path) — which the reader cannot use — to whichever of these exists. That one-line change is most of the value: it converts an unactionable warning into a pointer.

Bounded, obviously: the enumeration needs a cap and a *_truncated sibling like every other list here, and #205 is the fresh precedent for how (cap + always-present total, so "nothing cut" is a measurement).

What a fix must prove

With one partial file in the index and a query that does not name it, the reply's clause points at a call that actually returns that file.
Anti-vacuity, both arms: with zero partial files the enumeration returns an empty list as a measurement (not an absence, not a suppressed field), and a reply whose own body names the partial file still gets it inline in partial_sources as it does today.
An index that cannot be asked (older daemon, no active generation) reports unmeasured, not empty. partial_sources_unmeasured already models this; the new surface must match it rather than invent a second vocabulary.
The enumeration is capped, and the cap is exercised by a fixture that reaches it — #205's standard, for the same reason.

#216 (where partial_sources came from — this is its unfinished half), #205 (the bounding pattern any list surface should follow), #197 (the payload budget that rules out inlining), #148 (which planned the naming).

Filed 2026-09-09 against master 5ac45d9, observed live on code-index-mcp 0.26.1 (94d7de6).

Found by the #184/#205 lane, and independently confirmed by me across an entire working session — which is itself the evidence for the claim. ## The fact Every reply from this server on this repository carries: ```json "evidence_gaps": { "partial_sources": [], "partial_sources_in_index": 1, "semantics": "PARTIAL SOURCES: 1 file(s) … were produced by an extractor that reported an INCOMPLETE extraction … a count is a LOWER BOUND and an absent row is not evidence of absence … `index_coverage(path)` reports what each producer said." } ``` The three-state discipline here is correct and I am not disputing it. `partial_sources` names the partial files **this reply's own body names**; `partial_sources_in_index` is a `COUNT(DISTINCT path)` over the whole index. So when the partial file is not among your results, you get `1` and `[]`, and the design's own doc calls it exactly right: > it landed in `partial_sources_in_index` as **an anonymous denominator** and never in `partial_sources` as the named row ## The defect **The remedy the disclosure names requires the answer it is withholding.** `index_coverage(path)` reports what a producer said about *one path you already know*. The reader's question here is *which path*, and no surface answers it. The options available to a caller are: call `index_coverage` on every file in the repository, or ignore the warning. Both are the wrong answer for a disclosure that appears on **every single reply**. ## Why this is worse than a missing feature A warning that cannot be acted on is trained away. I read `partial_sources_in_index: 1` several dozen times in one session — while closing issues *about disclosure honesty* — and never once investigated it, because there was no next step. That is the failure mode: the disclosure is correct, permanent, and therefore invisible. It has become the wallpaper that `compose_evidence_semantics`'s own doc comment warns against: > A constant paragraph describing states this reply is not in is how a disclosure becomes wallpaper. The clause is not constant — it fires on a real condition — but from the reader's side it is indistinguishable from constant, because the condition never changes and nothing can be done about it. ## What must NOT be done - **Do not suppress it when `partial_sources` is empty.** That would collapse "one file in this index is partial and it is not in your results" into "nothing to report" — losing a true fact, and re-creating #216 exactly. - **Do not inline the whole list into every reply.** On a large repo the partial set could be long, and this pays a per-reply cost for a fact most callers do not need. #197's payload budget rules that out. - **Do not answer it with prose in the tool description.** Description prose is unread and expensive; this repo has measured that. - **Do not drop `partial_sources`' reply-scoped semantics.** The named-row behaviour is right — a partial file among *your* results deserves to be named inline. This issue is only about the anonymous remainder. ## Shape of a fix The cheapest honest thing is a **surface that enumerates them on demand**, so the per-reply clause can point at a call that works: - a resource (`code-index://index/partial-sources`), or - a field on `index_health` / `project_overview`, which already carry index-wide facts and are called once rather than per-query, or - an `index_coverage` mode that takes no path and returns the partial set. Then the `semantics` string's last sentence changes from `index_coverage(path)` — which the reader cannot use — to whichever of these exists. That one-line change is most of the value: it converts an unactionable warning into a pointer. Bounded, obviously: the enumeration needs a cap and a `*_truncated` sibling like every other list here, and #205 is the fresh precedent for how (cap + always-present total, so "nothing cut" is a measurement). ## What a fix must prove - With one partial file in the index and a query that does not name it, the reply's clause points at a call that **actually returns that file**. - **Anti-vacuity, both arms:** with zero partial files the enumeration returns an empty list as a *measurement* (not an absence, not a suppressed field), and a reply whose own body names the partial file still gets it inline in `partial_sources` as it does today. - An index that cannot be asked (older daemon, no active generation) reports **unmeasured**, not empty. `partial_sources_unmeasured` already models this; the new surface must match it rather than invent a second vocabulary. - The enumeration is capped, and the cap is exercised by a fixture that reaches it — #205's standard, for the same reason. ## Related #216 (where `partial_sources` came from — this is its unfinished half), #205 (the bounding pattern any list surface should follow), #197 (the payload budget that rules out inlining), #148 (which planned the naming). Filed 2026-09-09 against `master` `5ac45d9`, observed live on `code-index-mcp 0.26.1 (94d7de6)`.

dhoyer referenced this issue from a commit

2026-09-09 22:06:45 +02:00

fix: a disclosure may not name a remedy that cannot answer it (#243, #241)

dhoyer referenced this issue from a commit

2026-09-09 22:06:45 +02:00

merge lane/243+241: a disclosure names a call that terminates (#243, #241)

buildagent commented

2026-09-09 22:41:48 +02:00

Author

Member

Fixed on master at e97972c (merged 98d3e3f). The anonymous denominator now has a call that names it: resolution_gaps carries evidence_gaps.partial_source_census — sources (capped per project), total (exact, COUNT(DISTINCT path)), cap, truncated, unmeasured, semantics.

Both of this issue's suggested surfaces were wrong, and the measurements say why

A resource would have repeated the defect. MCP resources are not callable by the model in the clients this server targets — a user @-mentions them. Pointing an agent-facing disclosure at a resource hands the agent another next step it cannot take. Zero startup cost, zero reachability.

"A field on index_health / project_overview" is half-impossible. There is no index_health tool — it is a daemon RPC that project_overview composes. And project_overview has 3 tokens of content headroom, so it cannot carry a list at all.

And a new tool was ruled out by measurement, not preference: the startup payload sits at 16,503 of 16,515 — 12 tokens of an owed reserve. Median tool entry is 612 tokens. Even making index_coverage's path optional is ~+1 token of JSON schema, which breaches a reserve that "may not get worse".

resolution_gaps takes no required argument, is index-wide, is called deliberately rather than per query, and its unresolved counts are FLOORS because of partial sources — so the census belongs beside them. It is rendered from the PartialSourceProbe the grader already takes for every reply, so the census rows and partial_sources_in_index cannot disagree, and it costs zero extra queries.

Cost

path	delta
startup (`tools/list` + `initialize`)	0 — byte-for-byte identical
ordinary reply, repo with no partial sources	0
ordinary reply, repo with partial sources	+154 B (~38 tok), the reworded pointer
`resolution_gaps`	+131 tok

The +131 is attributed and re-recorded under the mechanism #235 shipped hours earlier — plugin-wpf 11786→11917 — and isolated by reverting one clause: with the census forced to None the tier measures 11786 exactly, so the whole move is the census and #243's hint rewrite costs zero there. It lands on one question; the other eighteen are byte-identical. 37 tokens were trimmed back before recording (the composed clause now points at the census rather than repeating its sentence).

Mutations

15 run, 15 RED, 0 survivors. The ones that decide it:

M7 — partial_source_census: None unconditionally → RED: "the call the clause names must ANSWER: resolution_gaps carried no partial_source_census.sources at all, so following the pointer lands the reader exactly where index_coverage(path) did." Note the lane's own honesty here: under M7 the composed clause still reads perfectly; only following it finds nothing. That is this issue in one line.
M8 / M11 — a clean index must SAY it is clean; sources: (total > 0).then(...) is the #216 shape and reds.
M10 — the unmeasured arm must not render as sources: [], total: 0: "there is no list to be empty when nobody was asked."
M12 — total from the page rather than the population → RED twice, unit and e2e at the 501-file cap.
M9 / M13 / M14 — the census must not ride on every reply; a reply that CARRIES the list must not send the reader away for it; an empty list must arrive with a sentence naming the block it is in.

The three-state discipline this issue insisted on is intact and graded in all three states.

Two gates caught the lane mid-work

evidence_semantics_registry G1 caught a first draft citing `evidence_gaps.partial_source_census` — which passed G3 only because the prefix stopped the token matching a field name: a citation answering to nothing, wearing a longer name. And disclosure_contract_e2e's clean-index rule needed resolution_gaps carved out as a measurement (asserting sources: [], total: 0, semantics) with a carved == 1 guard so the exemption cannot be for nobody.

An undisclosed limit found while working, worth its own issue

search_text on the exact phrase "search_text on its name confirms whether" returns total: 0 while that string is in server.rs — a Rust \-continued literal splits it across source lines, and the trigram index sees the raw bytes. Nothing in the reply discloses that a phrase can be defeated by a line continuation. Same class as the 3-character floor, which #52 does disclose. Filing separately.

Verification: fmt 0 · clippy -D warnings 0 · rustdoc 0 · cargo test --workspace 331 · both e2e legs 0 · corpus_ratchet baseline unmoved · precision_gate 7/7 phantoms=0.

Closing.

Fixed on `master` at `e97972c` (merged `98d3e3f`). The anonymous denominator now has a call that names it: **`resolution_gaps` carries `evidence_gaps.partial_source_census`** — `sources` (capped per project), `total` (exact, `COUNT(DISTINCT path)`), `cap`, `truncated`, `unmeasured`, `semantics`. ## Both of this issue's suggested surfaces were wrong, and the measurements say why **A resource would have repeated the defect.** MCP resources are not callable by the model in the clients this server targets — a *user* @-mentions them. Pointing an agent-facing disclosure at a resource hands the agent another next step it cannot take. Zero startup cost, zero reachability. **"A field on `index_health` / `project_overview`" is half-impossible.** There is no `index_health` *tool* — it is a daemon RPC that `project_overview` composes. And `project_overview` has **3 tokens** of content headroom, so it cannot carry a list at all. **And a new tool was ruled out by measurement, not preference:** the startup payload sits at 16,503 of 16,515 — **12 tokens** of an owed reserve. Median tool entry is 612 tokens. Even making `index_coverage`'s `path` optional is ~+1 token of JSON schema, which breaches a reserve that "may not get worse". `resolution_gaps` takes **no required argument**, is index-wide, is called deliberately rather than per query, and its `unresolved` counts are FLOORS *because of* partial sources — so the census belongs beside them. It is rendered from the `PartialSourceProbe` the grader already takes for every reply, so the census rows and `partial_sources_in_index` **cannot disagree**, and it costs zero extra queries. ## Cost | path | delta | |---|---| | startup (`tools/list` + `initialize`) | **0** — byte-for-byte identical | | ordinary reply, repo with no partial sources | **0** | | ordinary reply, repo with partial sources | +154 B (~38 tok), the reworded pointer | | `resolution_gaps` | **+131 tok** | The +131 is attributed and re-recorded under the mechanism #235 shipped hours earlier — `plugin-wpf` 11786→11917 — and **isolated by reverting one clause**: with the census forced to `None` the tier measures **11786 exactly**, so the whole move is the census and #243's hint rewrite costs zero there. It lands on one question; the other eighteen are byte-identical. 37 tokens were trimmed back before recording (the composed clause now *points at* the census rather than repeating its sentence). ## Mutations 15 run, 15 RED, 0 survivors. The ones that decide it: - **M7** — `partial_source_census: None` unconditionally → RED: *"the call the clause names must ANSWER: `resolution_gaps` carried no `partial_source_census.sources` at all, so following the pointer lands the reader exactly where `index_coverage(path)` did."* Note the lane's own honesty here: under M7 the composed clause still *reads* perfectly; only **following** it finds nothing. That is this issue in one line. - **M8 / M11** — a clean index must SAY it is clean; `sources: (total > 0).then(...)` is the #216 shape and reds. - **M10** — the unmeasured arm must not render as `sources: [], total: 0`: *"there is no list to be empty when nobody was asked."* - **M12** — `total` from the page rather than the population → RED twice, unit and e2e at the 501-file cap. - **M9 / M13 / M14** — the census must not ride on every reply; a reply that CARRIES the list must not send the reader away for it; an empty list must arrive with a sentence naming the block it is in. The three-state discipline this issue insisted on is intact and graded in all three states. ## Two gates caught the lane mid-work `evidence_semantics_registry` G1 caught a first draft citing `` `evidence_gaps.partial_source_census` `` — which passed G3 **only because the prefix stopped the token matching a field name**: a citation answering to nothing, wearing a longer name. And `disclosure_contract_e2e`'s clean-index rule needed `resolution_gaps` carved out **as a measurement** (asserting `sources: []`, `total: 0`, semantics) with a `carved == 1` guard so the exemption cannot be for nobody. ## An undisclosed limit found while working, worth its own issue `search_text` on the exact phrase `"search_text on its name confirms whether"` returns **`total: 0`** while that string is in `server.rs` — a Rust `\`-continued literal splits it across source lines, and the trigram index sees the raw bytes. Nothing in the reply discloses that a *phrase* can be defeated by a line continuation. Same class as the 3-character floor, which #52 *does* disclose. Filing separately. Verification: `fmt` 0 · `clippy -D warnings` 0 · rustdoc 0 · `cargo test --workspace` 331 · both e2e legs 0 · `corpus_ratchet` baseline unmoved · `precision_gate` 7/7 phantoms=0. Closing.

buildagent closed this issue

2026-09-09 22:42:02 +02:00

buildagent referenced this issue

2026-09-10 00:26:16 +02:00

Two calls seconds apart served DIFFERENT index generations while answer_provenance was byte-identical: nothing says which generation answered #245

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

h-dv/code-index#241

No description provided.

Rows
Columns

evidence_gaps warns on every reply that the answer may be short, and gives no way to find out which file — index_coverage(path) needs the path you are trying to learn #241