h-dv/code-index

Watch

Fork 0

bug: a metadata-only touch bricks changed_symbols + review_diff for up to 30 minutes, with a retry hint that cannot come true #82

New issue

Closed

opened 2026-08-26 13:00:34 +02:00 by buildagent · 1 comment

buildagent commented

2026-08-26 13:00:34 +02:00

Member

Found dogfooding v0.22.2 (4463d60). Live, reproduced end-to-end, pre-existing — not introduced by the reclassification fix in that commit (see "Not the new guard" below).

Reproduction

1. edit a tracked source file (content change)   → indexed, stale: false
2. touch it                                      → mtime bumps, content byte-identical
3. changed_symbols(base="HEAD")                  → ERROR index_updating
4. review_diff(base="HEAD")                      → ERROR index_updating
5. wait                                          → still bricked

md5 verified identical across step 2, and the file was indexed after its content change — so the index provably matches disk. Both tools refuse to answer about a correct index.

The error:

index_updating — "1 changed source file(s) are still pending in the index after the bounded wait. Retry the same call in a moment — no stale symbol ids/spans were returned."

Retrying does not help. It heals only when a content event arrives, or at the 30-minute periodic reconcile (watcher.rs:71). Confirmed both ways: repeated retries stayed bricked; restoring the file (a content event) healed it instantly.

Mechanism

Deliberate, and each step is individually reasonable:

watcher.rs:952 — EventKind::Modify(ModifyKind::Metadata(_)) => {}. Pure metadata edits (chmod / chown / touch) are dropped by design, labelled P1-2: "Without this filter, chmod -R on a 100k tree thrashes the writer with hash reads."
So files.mtime_ns is never refreshed for a metadata-only change.
file_freshness (crates/mcp-server/src/server.rs) computes staleness as disk_mtime != overlap.mtime_ns → index_stale: true.
The diff tools' freshness barrier waits on stale changed files, then errors index_updating.

Step 1 is right in isolation. Step 3 uses mtime as a proxy for "content may have changed", which the metadata filter has just made permanently false for this file. Nothing reconciles the two.

Why this isn't a manual-`touch` curiosity

The realistic triggers are ordinary:

chmod -R / chown -R anywhere in the tree
git checkout / git stash pop restoring byte-identical content
rsync -t, tar -p, cp -p — the exact overwrites ParseTask.ctime_ns was added to detect
build steps and codegen that rewrite identical bytes
an editor saving a file with no change

Any of those landing on a file that is also modified against the diff base bricks both tools for up to 30 minutes.

A fifth I034 door

I034's fix was walker-proved not_indexed for files that can never be indexed. This file can be indexed, is indexed, and is correct — the failure is in the mtime bookkeeping used as a freshness proxy. Different door, same outcome: the barrier waits for something that will not arrive.

Worse than I034 in one respect: the hint names a remedy that cannot work. I063 was about a disclosure asserting a cause nothing measured; this asserts a remedy nothing can deliver.

Not the new guard

4463d60 changed the hash-tier shortcut in run_task. It is not implicated: the watcher drops the metadata event at watcher.rs:952, before any of that code runs, and the classification here never changes (code stays code), so the guard's condition is satisfied and behaviour is byte-identical to before. Reproduced against the installed 4463d60 build; the mechanism is unchanged from every prior release.

Proposed fix

Two parts, both generic — no file type or trigger named.

1. Make the barrier's staleness check truthful, for the bounded set it gates on. The barrier only cares about files in the diff — a small, already-enumerated set. For those, when mtime differs, verify by content hash before blocking. That is the same evidence classify_hash already uses, applied where the answer changes behaviour. It costs one read per changed file and makes the barrier's claim true rather than probable. It does not touch the watcher's P1-2 filter, which remains correct for the 100k-file chmod -R case it was written for.

2. A hint that cannot be acted on is worse than none. If the barrier still gives up, it must not promise a retry. The honest text names the actual state and the actual remedies (touch the content, or wait for the periodic reconcile) — the index_coverage/not_indexed_reason discipline from I063, applied to a remedy instead of a cause.

Test shape

Registry-style, both directions:

content-changed file → barrier correctly waits, then heals (control — without this the fix could degenerate into "never block").
metadata-only touch on an otherwise-identical file → barrier does not block; both diff tools answer.
anti-vacuity: assert the file really is in the diff and really reports a newer mtime, or the second case passes for the wrong reason.
mutation: revert the hash verification → the metadata case must go red.

Runtime-plugin architecture requirement

#75 makes freshness two-dimensional: source bytes and active extraction generation. The immediate metadata-touch fix remains required, but it must not hard-code “fresh means file hash matches.”

For a diff file, the barrier’s usable-state predicate becomes:

disk content hash matches the indexed source snapshot;
the row/contributions belong to the project’s active generation;
no successful generation promotion/cache epoch is pending publication;
a requested but failed/rejected plugin is disclosed as coverage state, not watcher lag;
a building generation does not block tools that can truthfully answer from the still-active generation, unless the caller explicitly requests pending-generation semantics.

The bounded content-hash verification proposed above remains the correct repair for metadata-only touches. #78 must reuse it when validating source snapshots during generation builds, and #80 must pin distinct reason codes for source_pending, generation_pending, plugin_rejected and permanently_unclaimed.

Additional acceptance:

metadata-only touch under an active dynamic plugin does not block;
package generation promotion invalidates generation-bound cursors/caches even when source bytes are unchanged;
failed pending activation leaves diff tools usable against the previous active generation with disclosure;
no global plugin_set/is_code_path decision is used for linked projects with different package sets.

Found dogfooding v0.22.2 (`4463d60`). **Live, reproduced end-to-end, pre-existing** — not introduced by the reclassification fix in that commit (see "Not the new guard" below). ## Reproduction ``` 1. edit a tracked source file (content change) → indexed, stale: false 2. touch it → mtime bumps, content byte-identical 3. changed_symbols(base="HEAD") → ERROR index_updating 4. review_diff(base="HEAD") → ERROR index_updating 5. wait → still bricked ``` md5 verified identical across step 2, and the file was indexed *after* its content change — so **the index provably matches disk**. Both tools refuse to answer about a correct index. The error: > `index_updating` — "1 changed source file(s) are still pending in the index after the bounded wait. **Retry the same call in a moment** — no stale symbol ids/spans were returned." Retrying does not help. It heals only when a **content** event arrives, or at the 30-minute periodic reconcile (`watcher.rs:71`). Confirmed both ways: repeated retries stayed bricked; restoring the file (a content event) healed it instantly. ## Mechanism Deliberate, and each step is individually reasonable: 1. `watcher.rs:952` — `EventKind::Modify(ModifyKind::Metadata(_)) => {}`. Pure metadata edits (chmod / chown / touch) are dropped by design, labelled P1-2: *"Without this filter, `chmod -R` on a 100k tree thrashes the writer with hash reads."* 2. So `files.mtime_ns` is never refreshed for a metadata-only change. 3. `file_freshness` (`crates/mcp-server/src/server.rs`) computes staleness as `disk_mtime != overlap.mtime_ns` → `index_stale: true`. 4. The diff tools' freshness barrier waits on stale changed files, then errors `index_updating`. Step 1 is right in isolation. Step 3 uses mtime as a proxy for "content may have changed", which the metadata filter has just made permanently false for this file. Nothing reconciles the two. ## Why this isn't a manual-`touch` curiosity The realistic triggers are ordinary: - `chmod -R` / `chown -R` anywhere in the tree - `git checkout` / `git stash pop` restoring byte-identical content - `rsync -t`, `tar -p`, `cp -p` — the exact overwrites `ParseTask.ctime_ns` was added to detect - build steps and codegen that rewrite identical bytes - an editor saving a file with no change Any of those landing on a file that is *also* modified against the diff base bricks both tools for up to 30 minutes. ## A fifth I034 door I034's fix was walker-**proved** `not_indexed` for files that can never be indexed. This file **can** be indexed, **is** indexed, and is **correct** — the failure is in the mtime bookkeeping used as a freshness proxy. Different door, same outcome: the barrier waits for something that will not arrive. Worse than I034 in one respect: the hint names a remedy that cannot work. I063 was about a disclosure asserting a cause nothing measured; this asserts a *remedy* nothing can deliver. ## Not the new guard `4463d60` changed the hash-tier shortcut in `run_task`. It is not implicated: the watcher drops the metadata event at `watcher.rs:952`, **before** any of that code runs, and the classification here never changes (code stays code), so the guard's condition is satisfied and behaviour is byte-identical to before. Reproduced against the installed `4463d60` build; the mechanism is unchanged from every prior release. ## Proposed fix Two parts, both generic — no file type or trigger named. **1. Make the barrier's staleness check truthful, for the bounded set it gates on.** The barrier only cares about files *in the diff* — a small, already-enumerated set. For those, when mtime differs, verify by **content hash** before blocking. That is the same evidence `classify_hash` already uses, applied where the answer changes behaviour. It costs one read per changed file and makes the barrier's claim true rather than probable. It does **not** touch the watcher's P1-2 filter, which remains correct for the 100k-file `chmod -R` case it was written for. **2. A hint that cannot be acted on is worse than none.** If the barrier still gives up, it must not promise a retry. The honest text names the actual state and the actual remedies (touch the content, or wait for the periodic reconcile) — the `index_coverage`/`not_indexed_reason` discipline from I063, applied to a remedy instead of a cause. ## Test shape Registry-style, both directions: - **content-changed file** → barrier correctly waits, then heals (control — without this the fix could degenerate into "never block"). - **metadata-only touch on an otherwise-identical file** → barrier does **not** block; both diff tools answer. - **anti-vacuity:** assert the file really is in the diff and really reports a newer mtime, or the second case passes for the wrong reason. - **mutation:** revert the hash verification → the metadata case must go red. ## Runtime-plugin architecture requirement #75 makes freshness two-dimensional: source bytes and active extraction generation. The immediate metadata-touch fix remains required, but it must not hard-code “fresh means file hash matches.” For a diff file, the barrier’s usable-state predicate becomes: - disk content hash matches the indexed source snapshot; - the row/contributions belong to the project’s active generation; - no successful generation promotion/cache epoch is pending publication; - a requested but failed/rejected plugin is disclosed as coverage state, not watcher lag; - a building generation does not block tools that can truthfully answer from the still-active generation, unless the caller explicitly requests pending-generation semantics. The bounded content-hash verification proposed above remains the correct repair for metadata-only touches. #78 must reuse it when validating source snapshots during generation builds, and #80 must pin distinct reason codes for source_pending, generation_pending, plugin_rejected and permanently_unclaimed. Additional acceptance: - metadata-only touch under an active dynamic plugin does not block; - package generation promotion invalidates generation-bound cursors/caches even when source bytes are unchanged; - failed pending activation leaves diff tools usable against the previous active generation with disclosure; - no global plugin_set/is_code_path decision is used for linked projects with different package sets.

buildagent added the

labels

2026-08-26 13:00:38 +02:00

dhoyer referenced this issue from a commit

2026-08-26 13:12:10 +02:00

fix: a touch is not a change (#82)

buildagent referenced this issue

2026-08-26 13:41:21 +02:00

feat: review_diff v2 — source plus plugin-generation semantic risk #34

buildagent added a new dependency

2026-08-26 13:42:55 +02:00

#78 index: generation-safe plugin activation, invalidation, promotion and rollback

buildagent added a new dependency

2026-08-26 13:43:12 +02:00

#80 gate: plugin conformance, hostile-runtime containment and production disclosure

buildagent referenced this issue

2026-08-26 13:46:11 +02:00

index: generation-safe plugin activation, invalidation, promotion and rollback #78

buildagent referenced this issue

2026-08-26 13:46:17 +02:00

gate: plugin conformance, hostile-runtime containment and production disclosure #80

dhoyer referenced this issue from a commit

2026-08-26 14:03:22 +02:00

test: the #82 fix shipped without a test that could catch it

buildagent referenced this issue

2026-08-26 14:22:53 +02:00

bug: the client never compares the daemon's recorded version — a stale daemon serves degraded answers silently #83

dhoyer referenced this issue from a commit

2026-09-01 10:50:32 +02:00

fix: freshness is answered from the bytes, not from a timestamp (#80 S56)

dhoyer referenced this issue from a commit

2026-09-01 11:36:16 +02:00

chore: v0.23.0

buildagent commented

2026-09-04 11:38:54 +02:00

Author

Member

Fixed and graded — closing the bookkeeping

The fix is c807368 "fix: a touch is not a change (#82)", verified an ancestor of master and shipping since. Re-checked today rather than taken from a note.

The defect, as the test's own doc records it: mtime is a proxy for changed content, and the watcher deliberately drops metadata-only events — so a touch leaves the proxy permanently wrong. mtime moved, bytes did not, and nothing refreshes the row before the 30-minute reconcile. Measured live, that bricked changed_symbols and review_diff with index_updating ("retry in a moment") against an index that was provably correct.

Graded by a_touch_is_not_a_change_but_an_edit_still_is (crates/mcp-server/src/server.rs), and the name is the point: it carries both directions. Its doc says so explicitly —

Three arms, and the CONTROL is what stops the fix degenerating into "never report stale".

which is the failure this class invites: a freshness barrier that never reports staleness passes every "it did not brick" test and is useless. The control is what makes the fix a fix rather than a mute.

The barrier's neighbouring states are graded in the same module — a file that cannot be classified must be reported neither PENDING (the barrier polls that flag and then denies the whole tool when it never heals) nor permanently uncoverable ("a verdict about the user's file invented out of a daemon that could not answer").

Closing. This was linked from #80's adjacent-blocker list, so that list is one shorter.

## Fixed and graded — closing the bookkeeping The fix is `c807368` *"fix: a touch is not a change (#82)"*, verified an ancestor of master and shipping since. Re-checked today rather than taken from a note. The defect, as the test's own doc records it: `mtime` is a **proxy** for changed content, and the watcher deliberately drops metadata-only events — so a `touch` leaves the proxy permanently wrong. mtime moved, bytes did not, and nothing refreshes the row before the 30-minute reconcile. Measured live, that bricked `changed_symbols` and `review_diff` with `index_updating` ("retry in a moment") **against an index that was provably correct**. Graded by `a_touch_is_not_a_change_but_an_edit_still_is` (`crates/mcp-server/src/server.rs`), and the name is the point: it carries **both directions**. Its doc says so explicitly — > Three arms, and the CONTROL is what stops the fix degenerating into "never report stale". which is the failure this class invites: a freshness barrier that never reports staleness passes every "it did not brick" test and is useless. The control is what makes the fix a fix rather than a mute. The barrier's neighbouring states are graded in the same module — a file that cannot be classified must be reported neither PENDING (the barrier polls that flag and then denies the whole tool when it never heals) nor permanently uncoverable (*"a verdict about the user's file invented out of a daemon that could not answer"*). Closing. This was linked from #80's adjacent-blocker list, so that list is one shorter.

buildagent closed this issue

2026-09-04 11:39:00 +02:00

buildagent referenced this issue

2026-09-04 15:21:20 +02:00

gate: plugin conformance, hostile-runtime containment and production disclosure #80

buildagent referenced this issue

2026-09-05 09:10:26 +02:00

index_coverage reports "Indexed and current" for a file whose facts were refused by the validator #136

buildagent referenced this issue

2026-09-05 09:11:12 +02:00

audit: prove every permanent-refusal, plugin and generation coverage state #72

buildagent referenced this issue