h-dv/code-index

Watch

Fork 0

Two missing tools force a shell fallback: directory inventory, and read_code on a bare path #89

New issue

Closed

opened 2026-09-03 12:29:37 +02:00 by buildagent · 2 comments

buildagent commented

2026-09-03 12:29:37 +02:00

Member

From a customer session

Genuine tool gaps pushed me to shell. ls -la, find, wc -l — I used those to answer "what files are in this directory" and "how big are they". code-index has no directory-inventory tool. And read_code requires path:start-end or a symbol_id; when I didn't yet know the line range, sed -n '700,830p' was one call versus outline-then-read.

Their own proposal, which is the right shape:

a list_files(path_glob) returning path + line count + symbol count (kills ls/find/wc -l), and letting read_code(path) take a bare path with a size cap (kills cat/head).

Why this matters more than it looks

Neither gap is about answering a question WRONG — both are about the index having no answer at all, so the user opens a shell. The same report names what happens next: "Once I had a shell loop going, batching greps was lower friction than composing the right index query." A missing tool does not cost one call; it costs the rest of the session.

A — `list_files(path_glob)`

One row per file: path, line count, symbol count, lang, and whether it is symbol-blind. Everything is already in files/symbols; this is a query we do not expose.

It also answers a question search_text structurally cannot: what is HERE, as opposed to what matches this. And it should carry the coverage fact per row — a file that is indexed-as-text is a different answer from one that is not indexed at all, and today the user has to call index_coverage per path to learn it.

B — `read_code(path)` with a size cap

Today the target must be path:start-end or a symbol_id. Before you know the range, file_outline + read_code is two calls where sed -n is one. A bare path should serve the file up to the existing token cap and say so through the fields it already has (start_line, end_line, truncated), so a truncated read is a MEASUREMENT and not a silent prefix.

Bound worth stating in the reply

The customer also measured where the index is weak in their repo, and it is the honest limit rather than a defect: XAML and typed DataSets are symbol-blind, and C# reference resolution is 36%, so ref_count is a floor, not a measurement. They had to keep writing "ref_count 0 is not evidence of unused" by hand — which is exactly what our name_fallback_unmeasured / count_basis disclosures exist to say. Worth checking whether those disclosures actually reach a C#-heavy repo's rows, or whether the user is re-deriving something we already know.

## From a customer session > Genuine tool gaps pushed me to shell. `ls -la`, `find`, `wc -l` — I used those to answer "what files are in this directory" and "how big are they". **code-index has no directory-inventory tool.** And `read_code` requires `path:start-end` or a `symbol_id`; when I didn't yet know the line range, `sed -n '700,830p'` was one call versus outline-then-read. Their own proposal, which is the right shape: > a `list_files(path_glob)` returning path + line count + symbol count (kills `ls`/`find`/`wc -l`), and letting `read_code(path)` take a bare path with a size cap (kills `cat`/`head`). ## Why this matters more than it looks Neither gap is about answering a question WRONG — both are about the index having no answer at all, so the user opens a shell. The same report names what happens next: *"Once I had a shell loop going, batching greps was lower friction than composing the right index query."* **A missing tool does not cost one call; it costs the rest of the session.** ## A — `list_files(path_glob)` One row per file: path, line count, symbol count, `lang`, and whether it is symbol-blind. Everything is already in `files`/`symbols`; this is a query we do not expose. It also answers a question `search_text` structurally cannot: *what is HERE*, as opposed to *what matches this*. And it should carry the coverage fact per row — a file that is indexed-as-text is a different answer from one that is not indexed at all, and today the user has to call `index_coverage` per path to learn it. ## B — `read_code(path)` with a size cap Today the target must be `path:start-end` or a `symbol_id`. Before you know the range, `file_outline` + `read_code` is two calls where `sed -n` is one. A bare path should serve the file up to the existing token cap and say so through the fields it already has (`start_line`, `end_line`, `truncated`), so a truncated read is a MEASUREMENT and not a silent prefix. ## Bound worth stating in the reply The customer also measured where the index is weak in their repo, and it is the honest limit rather than a defect: **XAML and typed DataSets are symbol-blind, and C# reference resolution is 36%**, so `ref_count` is a floor, not a measurement. They had to keep writing "ref_count 0 is not evidence of unused" by hand — which is exactly what our `name_fallback_unmeasured` / `count_basis` disclosures exist to say. Worth checking whether those disclosures actually reach a C#-heavy repo's rows, or whether the user is re-deriving something we already know.

buildagent referenced this issue

2026-09-03 14:58:42 +02:00

honesty: no PROJECT-scoped asserted-name census (project_overview / resolution_gaps) and find_callees has no aggregate at any scope — the per-row marker and page count shipped in d0279c0 #93

buildagent commented

2026-09-03 18:05:18 +02:00

Author

Member

A third shell fallback, measured by a lane that used the tools all day

The #87.2 lane ran its entire investigation through search_symbols / file_outline / read_code / search_text and hit exactly one limit repeatedly:

search_text returns one row per file with up to 20 line numbers but not the matched lines' text, and has no alternation — so any "show me every site matching A|B across the tree" sweep still lands on grep.

Two distinct gaps, and the second is the one that forces the shell:

1. Line numbers without their lines. matches_in_file.lines gives up to 20 positions; reading any of them costs a read_code per hit. For a 10-hit file that is 10 round trips to answer a question grep -n answers in one. A matched_lines option — the line text beside each number, capped and truncated per line — would close it without changing the row shape.

2. No alternation. The trigram index is a substring matcher, so A|B cannot be expressed at all. Every "audit every site that says X or Y" sweep — which is how nearly every registry and gate in this repo is written — is a grep -E by construction. This is a real bound and may be the honest answer, but if so it should be stated in the tool's own semantics rather than discovered.

Why this belongs with #89. Both original gaps were about the index having no answer, so the user opened a shell. This is the same shape one level up: the index has the answer and hands back coordinates instead of content, and the shell is one call where we are ten. And the compounding cost is already recorded on this issue — "once I had a shell loop going, batching greps was lower friction than composing the right index query."

Filing here rather than separately because the fix lives in the same tool surface #89 just touched, and whoever picks it up should decide both together: list_files answered "what is here", and this answers "show me the hits, not where they are".

## A third shell fallback, measured by a lane that used the tools all day The #87.2 lane ran its entire investigation through `search_symbols` / `file_outline` / `read_code` / `search_text` and hit exactly one limit repeatedly: > `search_text` returns one row per file with up to 20 **line numbers** but not the matched lines' **text**, and has no alternation — so any "show me every site matching A|B across the tree" sweep still lands on `grep`. Two distinct gaps, and the second is the one that forces the shell: **1. Line numbers without their lines.** `matches_in_file.lines` gives up to 20 positions; reading any of them costs a `read_code` per hit. For a 10-hit file that is 10 round trips to answer a question `grep -n` answers in one. A `matched_lines` option — the line text beside each number, capped and truncated per line — would close it without changing the row shape. **2. No alternation.** The trigram index is a substring matcher, so `A|B` cannot be expressed at all. Every "audit every site that says X or Y" sweep — which is how nearly every registry and gate in this repo is written — is a `grep -E` by construction. This is a real bound and may be the honest answer, but if so it should be stated in the tool's own semantics rather than discovered. **Why this belongs with #89.** Both original gaps were about the index having *no answer*, so the user opened a shell. This is the same shape one level up: the index has the answer and hands back coordinates instead of content, and the shell is one call where we are ten. And the compounding cost is already recorded on this issue — *"once I had a shell loop going, batching greps was lower friction than composing the right index query."* Filing here rather than separately because the fix lives in the same tool surface #89 just touched, and whoever picks it up should decide both together: `list_files` answered "what is here", and this answers "show me the hits, not where they are".

dhoyer referenced this issue from a commit

2026-09-03 20:34:01 +02:00

feat: the infrastructure a language plugin needs, and a daemon that can be diagnosed

buildagent commented

2026-09-04 09:43:19 +02:00

Author

Member

Both tools shipped — verified live against the installed build

Directory inventory — list_files:

list_files(path_glob="crates/mcp-server/src/*")
-> 11 rows: path, lang, indexed_as, symbols, lines, bytes
   symbol-blind files carry `symbols_unmeasured: "symbol_blind"` rather than `symbols: 0`
   plus `not_indexed: []` and `not_indexed_scan: {directory, entries_capped: false, truncated: false}`

The not_indexed half is the part that makes it answer the actual question: it says what the index cannot see in that directory, as a measurement, so an empty list is "nothing hidden here" and not "we did not look".

read_code on a bare path:

read_code(target="crates/mcp-server/src/errors.rs")
-> start_line: 1, end_line: 153, total_lines: 153, truncated: false

total_lines is the file's measured length, so truncated: true would be a measurement rather than a silent prefix — total_lines - end_line is exactly what the cap withheld.

That closes the "two calls where sed -n was one" complaint this issue was filed about.

Related, from the same family and fixed in v0.26.1: a mistyped range (path:60,200, a comma for the hyphen) used to answer internal_error with a hint to check whether the daemon or index DB was down — an input typo reported as infrastructure. It now answers invalid_target with the syntax.

## Both tools shipped — verified live against the installed build **Directory inventory — `list_files`:** ``` list_files(path_glob="crates/mcp-server/src/*") -> 11 rows: path, lang, indexed_as, symbols, lines, bytes symbol-blind files carry `symbols_unmeasured: "symbol_blind"` rather than `symbols: 0` plus `not_indexed: []` and `not_indexed_scan: {directory, entries_capped: false, truncated: false}` ``` The `not_indexed` half is the part that makes it answer the actual question: it says what the index *cannot* see in that directory, as a measurement, so an empty list is "nothing hidden here" and not "we did not look". **`read_code` on a bare path:** ``` read_code(target="crates/mcp-server/src/errors.rs") -> start_line: 1, end_line: 153, total_lines: 153, truncated: false ``` `total_lines` is the file's measured length, so `truncated: true` would be a measurement rather than a silent prefix — `total_lines - end_line` is exactly what the cap withheld. That closes the "two calls where `sed -n` was one" complaint this issue was filed about. Related, from the same family and fixed in v0.26.1: a mistyped range (`path:60,200`, a comma for the hyphen) used to answer `internal_error` with a hint to check whether the daemon or index DB was down — an input typo reported as infrastructure. It now answers `invalid_target` with the syntax.

buildagent closed this issue

2026-09-04 09:43:30 +02:00

buildagent referenced this issue

2026-09-04 15:20:14 +02:00

gate: cap startup/tool-schema payload including plugin disclosures #71

buildagent referenced this issue

2026-09-04 15:20:50 +02:00

The startup payload ratchet reports one total, not the five categories that would make a trim targetable #111

buildagent referenced this issue

2026-09-04 16:34:55 +02:00

The startup payload ratchet reports one total, not the five categories that would make a trim targetable #111