An unchanged restart re-parses ~743 files and re-resolves for 3–4 minutes (zammad-dev: every idle restart) #358

Closed
opened 2026-10-12 19:04:59 +02:00 by buildagent · 3 comments
Member

Field report (zammad-dev, 2026-10-12, 0.36.1 / 876b78b5, de.h-dv.ruby 0.10.0, de.h-dv.vue 0.1.0)

idle_timeout_min = 30, and the daemon has shut down for idle 29 times since 2026-10-06, which is a normal working day with pauses. After each restart the initial reconciliation re-parses the same ~743 files, all of them "reclassified", and resolves again:

time parse reclassified resolve scope invalidated
06:05Z 743 743 251 s Scoped 0
12:50Z 802 743 142 s Full —
13:34Z 743 743 253 s Scoped 0
16:52Z 843 745 205 s Full 14786

The 06:05 and 13:34 runs had no file changes at all, yet each re-parsed 743 files and spent over 4 minutes in resolution. Log: /opt/zammad-dev/.code-index/daemon.log lines 10570, 10628, 10723 and 10903 ("index stage complete: reference resolution").

The disclosure works as intended: every answer carries index_state: reconciling and "a zero is not absence". In practice, though, every return after a pause means 3–4 minutes where all counts are floors and have to be checked by hand.

Hypothesis (unverified)

743 is close to the project's 761 .vue files (de.h-dv.vue/vue). The reclassification of package-claimed files may not be persisted, so a restart sees each one as "claimed differently than recorded" and re-extracts it, which then invalidates and re-resolves its references.

To check

  • Reproduce: index a fixture with .vue files and the vue package, stop the daemon, restart with no changes, and count reclassified and parsed files. The expectation is 0 and 0.
  • Find which stored fact is compared at startup (claim/owner/component per file) and why it differs from what was written.
  • The fix must make an unchanged restart parse 0 files. The test asserts that, with the mutation that re-introduces the re-parse run.

Workaround given to the reporter

[daemon] idle_timeout_min with a large value, e.g. 480. Note that 0 means the 30-minute default, not "never" (crates/daemon/src/main.rs, IDLE_TIMEOUT).

🤖 Generated with Claude Code

## Field report (zammad-dev, 2026-10-12, 0.36.1 / 876b78b5, de.h-dv.ruby 0.10.0, de.h-dv.vue 0.1.0) `idle_timeout_min = 30`, and the daemon has shut down for idle 29 times since 2026-10-06, which is a normal working day with pauses. After each restart the initial reconciliation re-parses the same ~743 files, all of them "reclassified", and resolves again: | time | parse | reclassified | resolve | scope | invalidated | |---|---|---|---|---|---| | 06:05Z | 743 | 743 | 251 s | Scoped | 0 | | 12:50Z | 802 | 743 | 142 s | Full | — | | 13:34Z | 743 | 743 | 253 s | Scoped | 0 | | 16:52Z | 843 | 745 | 205 s | Full | 14786 | The **06:05 and 13:34 runs had no file changes at all**, yet each re-parsed 743 files and spent over 4 minutes in resolution. Log: `/opt/zammad-dev/.code-index/daemon.log` lines 10570, 10628, 10723 and 10903 ("index stage complete: reference resolution"). The disclosure works as intended: every answer carries `index_state: reconciling` and "a zero is not absence". In practice, though, every return after a pause means 3–4 minutes where all counts are floors and have to be checked by hand. ## Hypothesis (unverified) 743 is close to the project's 761 `.vue` files (`de.h-dv.vue/vue`). The reclassification of package-claimed files may not be persisted, so a restart sees each one as "claimed differently than recorded" and re-extracts it, which then invalidates and re-resolves its references. ## To check - Reproduce: index a fixture with `.vue` files and the vue package, stop the daemon, restart with no changes, and count `reclassified` and parsed files. The expectation is 0 and 0. - Find which stored fact is compared at startup (claim/owner/component per file) and why it differs from what was written. - The fix must make an unchanged restart parse 0 files. The test asserts that, with the mutation that re-introduces the re-parse run. ## Workaround given to the reporter `[daemon] idle_timeout_min` with a large value, e.g. 480. Note that `0` means the 30-minute default, not "never" (`crates/daemon/src/main.rs`, `IDLE_TIMEOUT`). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Author
Member

The workaround in this issue does not work: [daemon] idle_timeout_min is a dead config key

The reporter tested it (2026-10-12): they set idle_timeout_min = 480 at 17:07Z, and a daemon spawned by code-index-mcp at 17:27Z still logged idle timeout reached; shutting down idle=1800s at 17:57:18Z.

Verified in the tree (train-037, same on 0.36.1):

  • DaemonConfig::idle_timeout_min (crates/indexer/src/config.rs) is parsed and read by nothing in production. Its only other occurrences are a config test and the CLI's config template.
  • The daemon takes its timeout only from --idle-min (crates/daemon/src/main.rs, default 30; 0 falls back to the same 30-minute IDLE_TIMEOUT).
  • Every spawner builds the command in lifecycle::daemon_command, which passes only --root and --db and no environment.
  • code-index doctor reports "no dead keys" for [watcher] only, so it says nothing about this one.

Fix together with the re-parse, in the next train:

  1. The daemon reads [daemon] idle_timeout_min (the CLI flag wins when given). Say what 0 means and keep it consistent across both.
  2. doctor covers [daemon] keys.
  3. Test: a daemon spawned through daemon_command with idle_timeout_min set in the manifest honours it. Run the mutation that ignores the key.

Interim workaround sent to the reporter (not tested on their setup): start the daemon by hand with --idle-min 480 before the MCP client attaches.

🤖 Generated with Claude Code

## The workaround in this issue does not work: `[daemon] idle_timeout_min` is a dead config key The reporter tested it (2026-10-12): they set `idle_timeout_min = 480` at 17:07Z, and a daemon spawned by `code-index-mcp` at 17:27Z still logged `idle timeout reached; shutting down idle=1800s` at 17:57:18Z. Verified in the tree (train-037, same on 0.36.1): - `DaemonConfig::idle_timeout_min` (`crates/indexer/src/config.rs`) is parsed and read by nothing in production. Its only other occurrences are a config test and the CLI's config template. - The daemon takes its timeout only from `--idle-min` (`crates/daemon/src/main.rs`, default 30; `0` falls back to the same 30-minute `IDLE_TIMEOUT`). - Every spawner builds the command in `lifecycle::daemon_command`, which passes only `--root` and `--db` and no environment. - `code-index doctor` reports "no dead keys" for `[watcher]` only, so it says nothing about this one. Fix together with the re-parse, in the next train: 1. The daemon reads `[daemon] idle_timeout_min` (the CLI flag wins when given). Say what `0` means and keep it consistent across both. 2. `doctor` covers `[daemon]` keys. 3. Test: a daemon spawned through `daemon_command` with `idle_timeout_min` set in the manifest honours it. Run the mutation that ignores the key. Interim workaround sent to the reporter (not tested on their setup): start the daemon by hand with `--idle-min 480` before the MCP client attaches. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Author
Member

Root-cause candidate (read from the code, not yet reproduced)

Found by dogfooding code-index 0.36.1 on train-037 (search_text "reclassified" → reconcile::classify_stat → snapshot::FileSnapshot::producer_moved → snapshot::load).

  1. crates/indexer/src/snapshot.rs::load reads the snapshot with SELECT f.path, …, p.package_key, c.lang FROM {PRODUCER_FROM}. PRODUCER_FROM is files f LEFT JOIN file_contributions fc ON fc.file_id = f.id AND fc.generation_id = ?1 …, so a file with several live contributions yields several rows.
  2. The loop stores them with out.insert(path, FileSnapshot { … producer }), so the last row wins.
  3. A .vue file with a <script> region has (at least) two live contributions:
    • the Vue package's own component, lang de.h-dv.vue/vue = files.lang;
    • the delegated region component from writer::write_delegated → contributions::ensure_delegated_component, lang typescript.
  4. informative_producer(lang = "de.h-dv.vue/vue", key = "de.h-dv.vue", component_lang = "typescript") returns None, because row_lang("typescript") != files.lang.
  5. producer_moved("de.h-dv.vue") reads None as builtin, so it returns true. classify_stat then answers Reclassified, and the file is fully re-extracted, which invalidates and re-resolves its refs.

That fits the field numbers: about 743 of 761 .vue files (those with a script region) are reclassified on every start with zero changes. Which row SQLite returns last is plan-dependent, so it may vary between files and builds.

The watcher's per-event lookup shares PRODUCER_FROM (doc comment above it), so the same ambiguity may affect watcher events on .vue files.

Fix direction (generic, any package with regions)

  • The producer of a FILE is the contribution whose component stamps files.lang, not "any contribution". Select that row (e.g. filter row_lang(c.lang) = f.lang in SQL, or fold the rows so one informative producer wins over None), never last-row-wins.
  • Test: a .vue fixture with a script region, indexed, daemon restarted with no change, asserts reclassified == 0 and parsed == 0. Run the mutation that restores last-row-wins.
  • Check whether #359 (Vue component attributed as influence on Ruby refs) involves the same multi-contribution join.

🤖 Generated with Claude Code

## Root-cause candidate (read from the code, not yet reproduced) Found by dogfooding code-index 0.36.1 on train-037 (`search_text "reclassified"` → `reconcile::classify_stat` → `snapshot::FileSnapshot::producer_moved` → `snapshot::load`). 1. `crates/indexer/src/snapshot.rs::load` reads the snapshot with `SELECT f.path, …, p.package_key, c.lang FROM {PRODUCER_FROM}`. `PRODUCER_FROM` is `files f LEFT JOIN file_contributions fc ON fc.file_id = f.id AND fc.generation_id = ?1 …`, so a file with **several live contributions yields several rows**. 2. The loop stores them with `out.insert(path, FileSnapshot { … producer })`, so **the last row wins**. 3. A `.vue` file with a `<script>` region has (at least) two live contributions: - the Vue package's own component, `lang de.h-dv.vue/vue` = `files.lang`; - the delegated region component from `writer::write_delegated` → `contributions::ensure_delegated_component`, `lang typescript`. 4. `informative_producer(lang = "de.h-dv.vue/vue", key = "de.h-dv.vue", component_lang = "typescript")` returns `None`, because `row_lang("typescript") != files.lang`. 5. `producer_moved("de.h-dv.vue")` reads `None` as `builtin`, so it returns `true`. `classify_stat` then answers `Reclassified`, and the file is fully re-extracted, which invalidates and re-resolves its refs. That fits the field numbers: about 743 of 761 `.vue` files (those with a script region) are reclassified on **every** start with zero changes. Which row SQLite returns last is plan-dependent, so it may vary between files and builds. The watcher's per-event lookup shares `PRODUCER_FROM` (doc comment above it), so the same ambiguity may affect watcher events on `.vue` files. ## Fix direction (generic, any package with regions) - The producer of a FILE is the contribution whose component stamps `files.lang`, not "any contribution". Select that row (e.g. filter `row_lang(c.lang) = f.lang` in SQL, or fold the rows so one informative producer wins over `None`), never last-row-wins. - Test: a `.vue` fixture with a script region, indexed, daemon restarted with no change, asserts `reclassified == 0` and parsed `== 0`. Run the mutation that restores last-row-wins. - Check whether #359 (Vue component attributed as influence on Ruby refs) involves the same multi-contribution join. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Author
Member

Released in v0.40.0 (https://git.h-dv.de/h-dv/code-index/releases/tag/v0.40.0, PR #388, merge a48fb945): a restart with nothing changed parses nothing, including with the Vue package, and [daemon] idle_timeout_min is honoured.

Released in v0.40.0 (https://git.h-dv.de/h-dv/code-index/releases/tag/v0.40.0, PR #388, merge a48fb945): a restart with nothing changed parses nothing, including with the Vue package, and `[daemon] idle_timeout_min` is honoured.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
h-dv/code-index#358
No description provided.