zammad-dev field report 4: deleted/renamed paths served for up to an hour; ids rotate across a promotion; path_glob case hint

buildagent commented

2026-10-14 21:13:01 +02:00

Member

From zammad-dev (abby), build 0.37.1 (8df22f69). Zammad/Rails + Vue fork; Ruby is owned by de.h-dv.ruby. Observed 2026-10-14 during a ~90-file branch refactor with many git rm/git mv. Reported over A1X; the text is quoted, not verified.

Deleted/renamed paths keep being served. After a git rm/git mv of ~15 Ruby/TS files (for example app/models/h_dv/mail_account.rb and config/routes/h_dv_mail.rb), search_text returned those paths for many minutes.
- A spec deleted in the morning (spec/lib/h_dv/mail_translations_spec.rb) was still returned about an hour later.
- The only signal was the per-row line_text_unavailable. There was no reply-level lagging or stale marker.
- The watcher plus the periodic reconcile should have dropped them. Needs a reproduction first; we asked for the daemon.log lines and the reply's index_freshness block. See also the open "Linux dynamic re-watch" note on index staleness.
Symbol ids rotate across reconciling → promotion_resolving.
- Ids taken from changed_symbols/file_outline returned symbol_not_found from check_rename/safe_delete a few minutes later.
- resolve_handle on the stable_handle worked.
- Generic candidate: id-taking tools accept a stable_handle, and symbol_not_found says when the generation changed since the id was minted.
review_diff reports tested Ruby code as untested_change. described_class.execute(...) in specs binds nothing. Same family as #335 and #337; tracked there.
resolution_gaps read 0% during a promotion.
- index_state rides on every tool reply (the index_state_of single producer), so it should have said promotion_resolving. We asked the reporter to confirm.
- v0.38 (#375) adds resolution_pending for the edit→resolve window.
Minor: path_glob is case-sensitive. app/frontend/*ticket*create* missed TicketCreateContent.vue. empty_population could hint at a case mismatch, as separator_scan does for separators.

Per the operator rule for issue reports: reproduce, find the class, look for synergies (items 1, 2 and 4 are all stale-answer disclosures across index changes), and only then decide on the smallest generic change.

From zammad-dev (abby), build 0.37.1 (8df22f69). Zammad/Rails + Vue fork; Ruby is owned by de.h-dv.ruby. Observed 2026-10-14 during a ~90-file branch refactor with many `git rm`/`git mv`. Reported over A1X; the text is quoted, not verified. 1. **Deleted/renamed paths keep being served.** After a `git rm`/`git mv` of ~15 Ruby/TS files (for example `app/models/h_dv/mail_account.rb` and `config/routes/h_dv_mail.rb`), `search_text` returned those paths for many minutes. - A spec deleted in the morning (`spec/lib/h_dv/mail_translations_spec.rb`) was still returned about an hour later. - The only signal was the per-row `line_text_unavailable`. There was no reply-level lagging or stale marker. - The watcher plus the periodic reconcile should have dropped them. **Needs a reproduction first**; we asked for the `daemon.log` lines and the reply's `index_freshness` block. See also the open "Linux dynamic re-watch" note on index staleness. 2. **Symbol ids rotate across `reconciling` → `promotion_resolving`.** - Ids taken from `changed_symbols`/`file_outline` returned `symbol_not_found` from `check_rename`/`safe_delete` a few minutes later. - `resolve_handle` on the `stable_handle` worked. - Generic candidate: id-taking tools accept a `stable_handle`, and `symbol_not_found` says when the generation changed since the id was minted. 3. **`review_diff` reports tested Ruby code as `untested_change`.** `described_class.execute(...)` in specs binds nothing. Same family as #335 and #337; tracked there. 4. **`resolution_gaps` read 0% during a promotion.** - `index_state` rides on every tool reply (the `index_state_of` single producer), so it should have said `promotion_resolving`. We asked the reporter to confirm. - v0.38 (#375) adds `resolution_pending` for the edit→resolve window. 5. **Minor:** `path_glob` is case-sensitive. `app/frontend/*ticket*create*` missed `TicketCreateContent.vue`. `empty_population` could hint at a case mismatch, as `separator_scan` does for separators. Per the operator rule for issue reports: reproduce, find the class, look for synergies (items 1, 2 and 4 are all stale-answer disclosures across index changes), and only then decide on the smallest generic change.

buildagent commented

2026-10-14 21:21:47 +02:00

Author

Member

Item 1: reproduced and root-caused on 0.37.2. The fix goes into 0.38.0 (operator decision).

Repro:

Index a repo, then promote a plugin generation, for example with code-index languages owner ruby package. The previous generation is now superseded and holds a contribution for every file.
git rm some pre-promotion files and commit.

After the re-walk (and also after a daemon restart):

files and files_fts still hold the deleted paths.
search_text returns them, list_files lists them, project_overview.files counts them, and file_outline calls them indexed.
A git mv yields both the old path and the new one.

A file created after the promotion and then deleted is removed within 3 s, which is why a builtin-only repo does not reproduce it.

index_coverage → path_not_found comes from the disk check. The files row is still present.

Root cause:

writer.rs write_delete: when a superseded generation holds the file, it only withdraws the live rows. The anchor and its FTS text are kept for rollback until that generation is collected, which happens at the next promotion.
File-level readers do not check liveness. Only freshness::compare was given the live-contribution rule, in #327.
Every re-walk resends the delete, takes the same branch, and reports deleted=0.

Fix (lane F): a file gone from disk leaves the index entirely, including what the superseded generation retained. Restoring a deleted file's rows on rollback is wrong anyway; if the file returns, it is re-extracted. Existing ghost anchors are removed by the next reconcile or re-walk after upgrading. The re-walk log will report how many paths it removed.

Side finding: distribution/registry.v1.json points de.h-dv.ruby 0.10.1 at …/download/pkg%2Fde.h-dv.ruby%2Fv0.10.1/…, which returns 404. The package-only release has not been made yet; the asset exists only under the v0.37.x releases.

**Item 1: reproduced and root-caused on 0.37.2. The fix goes into 0.38.0 (operator decision).** **Repro:** 1. Index a repo, then promote a plugin generation, for example with `code-index languages owner ruby package`. The previous generation is now `superseded` and holds a contribution for every file. 2. `git rm` some pre-promotion files and commit. After the re-walk (and also after a daemon restart): - `files` and `files_fts` still hold the deleted paths. - `search_text` returns them, `list_files` lists them, `project_overview.files` counts them, and `file_outline` calls them indexed. - A `git mv` yields both the old path and the new one. A file created after the promotion and then deleted is removed within 3 s, which is why a builtin-only repo does not reproduce it. `index_coverage` → `path_not_found` comes from the disk check. The `files` row is still present. **Root cause:** - `writer.rs` `write_delete`: when a superseded generation holds the file, it only withdraws the live rows. The anchor and its FTS text are kept for rollback until that generation is collected, which happens at the next promotion. - File-level readers do not check liveness. Only `freshness::compare` was given the live-contribution rule, in #327. - Every re-walk resends the delete, takes the same branch, and reports `deleted=0`. **Fix (lane F):** a file gone from disk leaves the index entirely, including what the superseded generation retained. Restoring a deleted file's rows on rollback is wrong anyway; if the file returns, it is re-extracted. Existing ghost anchors are removed by the next reconcile or re-walk after upgrading. The re-walk log will report how many paths it removed. **Side finding:** `distribution/registry.v1.json` points de.h-dv.ruby 0.10.1 at `…/download/pkg%2Fde.h-dv.ruby%2Fv0.10.1/…`, which returns 404. The package-only release has not been made yet; the asset exists only under the v0.37.x releases.

buildagent referenced this issue from a commit

2026-10-14 23:11:53 +02:00

#376 item 1: a file gone from disk leaves the index entirely

buildagent referenced this issue from a commit

2026-10-14 23:11:53 +02:00

#376: a removal withdraws only a file still spelled that way

buildagent referenced this issue from a commit

2026-10-14 23:11:53 +02:00

Merge lane F (#376: a file gone from disk leaves the index entirely) into train-0380

buildagent referenced this issue

2026-10-14 23:12:10 +02:00

Release 0.38.0: the index DB is a rebuildable cache (#374), deleted files leave the index (#376), pending-resolve disclosure and lang hints (#375) #377

buildagent referenced this issue from a commit

2026-10-15 02:24:27 +02:00

Merge pull request 'Release 0.38.0: the index DB is a rebuildable cache (#374), deleted files leave the index (#376), pending-resolve disclosure and lang hints (#375)' (#377) from train-0380 into master

buildagent commented

2026-10-15 04:28:00 +02:00

Author

Member

Item 1 (deleted or renamed files kept being served after a plugin generation switch) is fixed in v0.38.0 (PR #377).

A file gone from disk now leaves the index entirely, including its retained rollback rows (WriterMsg::Delete).
A file still on disk but no longer walked keeps its rollback rows (WriterMsg::Withdraw).
Exact-case spelling is checked, so case-insensitive filesystems decide correctly.
Existing ghosts are removed on the first reconcile, and the re-walk log reports removed=N.

Item 3 is tracked in #337. Items 2, 4 and 5 remain open here.

Item 1 (deleted or renamed files kept being served after a plugin generation switch) is fixed in [v0.38.0](https://git.h-dv.de/h-dv/code-index/releases/tag/v0.38.0) (PR #377). - A file gone from disk now leaves the index entirely, including its retained rollback rows (`WriterMsg::Delete`). - A file still on disk but no longer walked keeps its rollback rows (`WriterMsg::Withdraw`). - Exact-case spelling is checked, so case-insensitive filesystems decide correctly. - Existing ghosts are removed on the first reconcile, and the re-walk log reports `removed=N`. Item 3 is tracked in #337. Items 2, 4 and 5 remain open here.

buildagent referenced this issue from a commit

2026-10-15 19:26:21 +02:00

#379: five replies say what the index state actually is

buildagent referenced this issue

2026-10-16 20:55:47 +02:00

Watcher batches wait 20 s+ behind a resolve on zammad-dev: a waiting batch does not preempt it (#327), so a deleted file stays served #390

~~buildagent referenced this issue 2026-10-21 19:43:41 +02:00~~

fix(runtime): resolve durable handles and refresh influence on upgrade #405

buildagent referenced this issue

2026-10-21 23:03:25 +02:00

fix(runtime): resolve durable handles and refresh influence on upgrade #405

buildagent commented

2026-11-09 09:21:46 +01:00

Author

Member

Triage 2026-11-09: FIXED: item 1 in b989cf69/3e84d9fa; item 2 (check_rename/safe_delete accept stable_handle); item 3 → #538 (merged #337); item 4 via resolution_pending/promotion_resolving; item 5 path_glob case hint (e3c5e4af, probed live). The generic 'ids can change' hint is a P4 nicety, noted in #521's friction list.

Triage 2026-11-09: FIXED: item 1 in b989cf69/3e84d9fa; item 2 (`check_rename`/`safe_delete` accept `stable_handle`); item 3 → #538 (merged #337); item 4 via `resolution_pending`/`promotion_resolving`; item 5 path_glob case hint (e3c5e4af, probed live). The generic 'ids can change' hint is a P4 nicety, noted in #521's friction list.

buildagent closed this issue

2026-11-09 09:21:46 +01:00

Rows
Columns

zammad-dev field report 4: deleted/renamed paths served for up to an hour; ids rotate across a promotion; path_glob case hint #376