Every agent lane forfeits symbol search for the code it just wrote: worktrees live under .claude/, which is permanently unindexable #238
Labels
No labels
code-review
correctness
dos
performance
security
severity/high
severity/low
severity/medium
tech-debt
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
h-dv/code-index#238
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Split out of #201, which is now closed. #201 stated two problems and both are fixed; this is the part that survived, and it is the one the original reporter actually cared about — "it forced that lane onto
grepfor code it had just written."The structural fact
Worktrees in this project are created under
.claude/worktrees/:.claudeis a dot-directory, and the walker allowlists only.github,.gitlaband.forgejo. Measured, with the tool's own proof:613 files with indexable extensions in a single lane worktree, times however many lanes are live (2 right now, routinely more).
What works and what does not — the split matters
read_codeworks. Verified live: an absolute path into a lane worktree routes to primary by longest root prefix and reads the bytes off disk.read_codeserves source bytes and does not consult the index, so the dot-directory rule never bites it.search_symbols,search_text,file_outline, and every graph tool do not, because they read rows, and there are no rows.So a lane can read a file it already knows the path of, and cannot find anything. That is the wrong half to have working for an agent that just wrote 600 files of new code.
Why this is ours and not a workflow complaint
CLAUDE.md makes preferring the index a hard rule that overrides the bypass-permissions guidance, and states the measured reason (1.00 precision vs ripgrep's 0.52, a quarter of the round trips). Every lane we spawn is then placed somewhere that rule cannot be followed, and the tool correctly tells it to use the shell. We are instructing agents to use an instrument and then standing them where it does not reach.
The honesty is genuinely good here —
verdict: neverwith the rule that fired and ahintnaming both remedies is exactly right, and this issue is not a complaint about the disclosure. It is that the disclosure is telling us something true about our own workflow.What must NOT be done
.claude/worktrees/. Each worktree is a near-complete copy of the repo. With N lanes live, every symbol in the project fans out N+1 times,ref_countbecomes meaningless, andsearch_symbolsreturns N copies of every hit. This would be strictly worse than not indexing them..claude/. That is the harness's directory choice, not ours to depend on, and a fix that works only while a third party keeps a path convention is not a fix.Shape of a fix
The one shape that survives all four constraints is explicit, disclosed, per-session linking: a worktree becomes a named project (not part of primary), reachable by
project=like any other link, with its identity inanswer_provenance.indexed_trees— which already exists and already reportsheadper tree, so the #182 requirement is met by machinery in the tree today.Open questions the fix has to answer, not assume:
.code-index.toml, or the server detecting that a[[links]]candidate is a worktree of primary?project=is the conservative reading, and matches how links already behave when a caller does not ask.What a fix must prove
search_symbolsa symbol it created in its own worktree, and the reply names which tree answered.ref_count.head, per #182.Related
#201 (closed; this is its residual), #182 (which tree produced the answer), #33 (the CI dot-dir allowlist, which is the precedent for allowlisting — and the precedent for why a blanket allowlist was not what #33 did), #191.
Found by dogfooding during the #228 integration, 2026-09-09, against
code-index-mcp 0.26.1 (94d7de6).Independent second hit, and a cheaper fix than the one this issue proposes
The #204 lane hit this from inside a worktree while I was filing it, without knowing this issue existed. Same
verdict: "never", reason: "hidden", proof.at: ".claude". Two independent encounters in one afternoon, which is the frequency argument this issue was missing.Their report adds a sharper framing than mine and a cheaper fix, so both go here.
The sharper framing: it is not that search fails, it is that it answers confidently
That sentence is true and misleading at the same time, which is the worst combination this project has a name for. "Measured across everything indexed" is accurate; nothing in
empty_populationsays the tree you are editing is not in any indexed root. A caller reads a confident measured zero for a file they just wrote.This is the ordering from #220, one tool over: a blind spot that reports as a measurement is worse than one that reports as a limit. #220 fixed that for
read_code's error prose;search_symbols'sempty_populationstill has it.The cheaper fix, and it may make the per-session linking unnecessary for most of the pain
I proposed disclosed per-session linking, which is real work. The lane points out most of the damage is done by the silence, not the absence:
That is right, and it splits this issue cleanly into two:
empty_populationnames the indexed roots, or carries the walker proofindex_coveragealready produces. An agent then knows in one call to fall back to the shell, instead of concluding the symbol does not exist. This is the disclosure half and it needs no indexing decision.Do (1) first. It is small, it is the same mechanism
index_coveragealready implements, and it converts the current failure from wrong answer to correct refusal with a next step. The lane's closing line is the argument: "This is why I usedgrepthroughout; that was the correct fallback, not a preference." Today an agent can only learn that by luck.One more datum from their run
Their read-only queries about committed code were correct, because they were based on the same commit the index had read (
answer_provenance.indexed_trees.primary.head: 915c85087b9f,uncommitted_changes: false). They note the hazard that follows:So the failure is not uniform. A worktree at the same commit as primary gets right answers about committed code and wrong answers about its own new code; a worktree at a different commit gets silently wrong answers about both.
answer_provenancealready carries theheadneeded to detect this — it is reported, just not compared against anything the caller is working in.Part 1 fixed on
masteratda7b34a. First, a correction to this issue, and the error is mine.I measured against a stale binary and over-scoped the defect
I filed this from live probes against the installed MCP server. Verified just now:
6c454ebis #220, which already attachesanswer_provenancetosymbol_not_foundand already shipssibling_worktrees: n. It is in the tree and not in the binary I probed. So the "confident measured zero with nothing at all to qualify it" I reported was a fixed defect, and anyone reading this issue would have over-scoped the work.This is the staleness trap I have spent today closing on issue text, arriving through the instrument instead. An open issue is a dated snapshot of the code; a running binary is a dated snapshot too, and I checked the first and not the second. I have re-verified the rest of today's live claims against
94d7de6:730bf15(the #202/#201/#149 payload-honesty work) is in it, so those confirmations stand. This issue is the only one affected.What the real residual was — one sentence
After #220 the reply carried the qualifying integer, but
empty_population.notestill asserted "a measured absence across everything indexed" while a sibling block said the population was incomplete. The qualifier lived in a different object from the claim it qualifies — and that sibling block's documented subject is which binary and which commit answered, not what was searched.So the fix is: the sentence that makes the universal claim carries its own exception, in the same object.
probe_empty_populationtakes anunsearchedparameter; when present the quantifier narrows from "across everything indexed" to "across the indexed roots named inunsearched, which are not every checkout of them".EmptyPopulation::unsearchednames each searched root by path. The oldhintsaid "Searched: primary" — and a caller cannot compare their own cwd against the wordprimary.UnsearchedTrees::from_roots(errors.rs:236) is the anti-vacuity rule as a pure function:Noneunless some root has a checkout the walk did not enter, or could not be asked about.tree_state's cache — the same entryanswer_provenancefills for the same reply — so no extragitspawn, and a successful call never reaches it.It generalises, measured rather than argued
grep "measured absence\|everything indexed"overserver.rsfinds exactly two sites that assert the universal, and both are insideprobe_empty_population— one function serving bothsearch_symbolsandsearch_text. One clause, two tools, and each of the four legs (fan-out andproject-pinned, per tool) separately mutation-graded.find_callers,find_references,explain_dependencytake asymbol_id, which only a prior indexed hit can mint — you cannot reach them for unindexed code.file_outlinereturnsdid_you_mean: ["primary=… — where … was resolved"], andlist_filesreturnstotal: 0with every entry innot_indexedcarryingreason: "hidden".So my comment's claim that "
search_symbols,search_text,file_outline, and every graph tool do not [work]" was true about rows and misleading about honesty. Only the two name-shaped tools were lying.Why not "is the caller inside an indexed root?" — it cannot be built.
readlink /proc/<pid>/cwdon both livecode-index-mcpprocesses returns the primary root while the caller sits in a worktree, and the server implements norootscapability, so no channel carries the caller's directory.Cost — zero on every common path
Byte-for-byte against a binary built from
5ac45d9(git archiveinto a separate tree, noCARGO_TARGET_DIR), same fixture, same probe:search_symbolssearch_textThe earned case is +651 bytes, once, only on a miss, only where a checkout provably exists — trimmed once after first measuring at +764. The token ratchet stayed green.
Mutations — 11 run, 0 survivors
I re-ran M1 myself, since it is the arm that decides whether this is a disclosure or wallpaper.
from_rootsdisclosing unconditionally:Restored by
cp, md5 verified, re-run11 passed; 0 failed.The other ten cover: a measured
0counted as unsearched (M2);unmeasuredcollapsing intomeasured zero(M3); a key growing on a clean reply (M4); the qualifier living only in a sibling block — "the #220 defect one field over" (M8); and a filtered zero attracting a worktree paragraph that explains nothing (M6).M9–M11 exist because the lane's first draft left them ungraded: the shared-clause claim rested on one tool and one leg. Arms added, then mutated to prove they bite. That is the right instinct — a claim that one clause serves two tools is not evidence until both tools are graded.
Rejected, with the reason
git worktree list --porcelainis already run and already cached, and also printsHEAD <oid>per entry, so the block could say "K of the N unsearched trees are at a different commit". Rejected: it changesSiblingWorktrees, a shipped #220 wire shape, for a fact that does not change the caller's next action (fall back to shell either way), and it costs bytes on the earned path. It belongs to part (2), where the head genuinely disambiguates which tree answered.A doc defect found and fixed in passing
docs/answer-provenance.mdtold readers to runcode-index link add <path-to-your-worktree>. The CLI iscode-index link add <name> <path>. The advertised remedy for this very issue did not run as written.Follow-up candidate, not fixed
file_outline'sno_outlinehint says "If the file was just created, the index may not have caught up" for a path the walker permanently refuses — framing a permanent exclusion as transient. #220 deliberately keepsno_outlineout ofMEASUREMENT_KEYS, so this is a scope call rather than an oversight.Gates
fmt0 ·clippy -D warnings0 ·cargo doc --document-private-itemsunderRUSTDOCFLAGS="-D warnings"0 ·cargo test --workspaceexit 0 (329 ok blocks) ·COSI_E2E_LEG=daemonexit 0 (62 ok blocks) — all four new tests pass on the daemon leg too.tests/corpus/baseline.jsonuntouched. Post-merge by me:zero_basis_e2e11 passed,answer_provenance_e2e18 passed, both EXIT=0.Part 1 is done; this issue stays open for part 2 (making worktrees reachable as named projects), which is unchanged and still constrained by all four "do not" items.
no_outlineandpath_not_indexedframe a PERMANENT exclusion as transient, and send the caller to a tool that cannot answer either #243Part 2 done on
masteratd4f121a(mergedde98a4f). Both halves of this issue are now closed.The shape, and how each of the four constraints is met structurally
COSI_WORKTREES_DIRnames a directory. Each child that proves it is a worktree of this repository — its.gitfile'sgitdir:points into this repo's ownworktrees/— becomes a linked project..claude/worktrees/ref_countis untouchedanswer_provenance.indexed_treesnames each tree with its ownhead.claude/worktreesappears nowhere in the codeThe three open questions, answered rather than assumed:
COSI_CORPUS_DIR/#191 precedent that argued through the same constraints.Relationship::Worktreecarries the rule, so it attaches to the fact (a second checkout of one repo) rather than to how the link was made. Disclosed inprojects_excluded_from_fan_outon success, and named in thehinton a miss.<worktree>/.code-index/index.db.Verified live, from inside a lane
search_symbols("render_link_package_set_semantics", project="agent-a268e9b8e3d9cc8ee")returned a function the lane had just written and not committed, at its real line.project="primary"and the fan-out both returnedsymbol_not_found.answer_provenance.indexed_treesnamed both trees with their own heads anduncommitted_changes: truefor the lane againstfalsefor primary.That is this issue's original complaint — "it forced that lane onto
grepfor code it had just written" — answered end to end.I verified the constraint that matters, and my first attempt was the wrong mutation
The rule most at risk is "do not depend on a path convention", so I ran it. Defaulting the env var to a bare relative
.claude/worktreesSURVIVED — because that resolves against the server's CWD, not the fixture root, so it found nothing. Redone root-relative:A mutation one construction off tests a different thing and looks like a refutation. Restored by
cp, md5 verified.The lane hit the same trap first and recorded it in the test's own doc: its original fixture put every worktree in
<tmp>/lanes, so a default pointing at<root>/.claude/worktreesfound nothing — "the test's NAME claimed a constraint its POPULATION could not reach." It now plants a real bait worktree in the most plausible default location, plus the reciprocal arm proving the bait is reachable when the operator asks.Two more mutation results worth reading
worktree_linkslinks nothing) reddened 7 tests and exposed a vacuity in the lane's own new test:the_two_trees_do_not_leak_into_each_otherstayed GREEN, because with no link there is nothing to leak. A reachability precondition was added; M3 now reddens it.Two defects found and fixed on the way
empty_population.unsearched.searched_rootspromises "every indexed root this query was answered from" and was listing the worktree the fan-out had skipped — in the same reply whosehintsaidSearched: primary. Now scoped to the consulted set.project_not_available, whose hint says "configured in[[links]]" — false twice. Shared withcorpus_links.Known limit, disclosed rather than hidden
The scan runs once at startup, so a worktree created later in the session is not linked. That gap is not silent: part 1's
empty_population.unsearchedstill names the roots it did search, and the link description says so in as many words.Verification:
fmt0 ·clippy -D warnings0 · rustdoc-D warnings0 ·cargo test --workspace336 suites, 0 failed ·COSI_E2E_LEG=daemon67 suites, 0 failed — the headline test runs on the configured leg per I035.baseline.jsonandratchet.jsonuntouched. Post-merge by me:worktree_projects_e2e11 passed,link_payload_scaling_e2e3 passed.Closing.
answer_provenancenames a commit the INDEX has not reached, so a watcher-lag miss and a measured absence are indistinguishable #260