Field (zammad-dev, 9.9k files): every daemon restart after idle timeout re-enters a full reconcile, answers stay floors, and per-language resolution reads stale until a resolve pass completes #326

Closed
opened 2026-10-07 08:59:31 +02:00 by buildagent · 3 comments
Member

Field report from zammad-dev (A1X code-index-dev, 2026-10-07) on a Zammad 7.0.2 fork at /opt/zammad-dev, about 9.9k files, running v0.33.1 (74f45e7b). Not yet reproduced here.

Observed

  • After an idle-timeout restart (pid 540297 → 549695), project_overview reports state=reconciling and "no resolve transaction has run yet in this daemon", with resolve_progress pass_seq=0.
    • This lasts "long stretches". Every restart appears to start a full reconcile.
    • While it lasts, every answer is a floor, so an empty result cannot be trusted.
  • Ruby ownership had been switched to the package (language_ownership: package_owns, de.h-dv.ruby). Ruby resolution then reported 4.6% of 361k refs, with method_call at 0.5% / 5.3% and read at 0.8%. find_callers and change_impact were mostly empty for Rails code.
    • These may be stale numbers from before the owner switch, because no resolve pass had run in that daemon.

To answer

  1. Does a restart of a daemon whose index is already current really need a full reconcile? If it does, how long does it take on a tree this size?
  2. Is #301's resumable resolution actually resuming after an idle-timeout restart, or starting again at pass_seq=0?
  3. Should per-language resolution be reported, or marked stale, while pass_seq=0 after an owner switch?
  4. After one resolve pass completes, is Ruby package resolution on a Rails app actually around 5%? If so, that is a real recall finding. The tier-1 corpus has no Rails app.

I asked the reporter for the reconcile duration and the restart cause.

Field report from zammad-dev (A1X code-index-dev, 2026-10-07) on a Zammad 7.0.2 fork at /opt/zammad-dev, about 9.9k files, running v0.33.1 (74f45e7b). **Not yet reproduced here.** ## Observed - After an idle-timeout restart (pid 540297 → 549695), `project_overview` reports `state=reconciling` and "no resolve transaction has run yet in this daemon", with `resolve_progress pass_seq=0`. - This lasts "long stretches". Every restart appears to start a full reconcile. - While it lasts, every answer is a floor, so an empty result cannot be trusted. - Ruby ownership had been switched to the package (`language_ownership: package_owns`, de.h-dv.ruby). Ruby resolution then reported 4.6% of 361k refs, with method_call at 0.5% / 5.3% and read at 0.8%. `find_callers` and `change_impact` were mostly empty for Rails code. - These may be stale numbers from before the owner switch, because no resolve pass had run in that daemon. ## To answer 1. Does a restart of a daemon whose index is already current really need a full reconcile? If it does, how long does it take on a tree this size? 2. Is #301's resumable resolution actually resuming after an idle-timeout restart, or starting again at `pass_seq=0`? 3. Should per-language resolution be reported, or marked stale, while `pass_seq=0` after an owner switch? 4. After one resolve pass completes, is Ruby package resolution on a Rails app actually around 5%? If so, that is a real recall finding. The tier-1 corpus has no Rails app. I asked the reporter for the reconcile duration and the restart cause.
Author
Member

Reporter follow-up with daemon.log data, 2026-10-07. This narrows the issue.

Item 1 (reconcile after restart): mostly withdrawn.

  • The restart came from the idle timeout ("idle timeout reached; shutting down idle=1800s" at 06:05:15Z).
  • The new daemon started at 06:54:21Z. "initial reconciliation complete; watcher armed" followed at 06:54:35Z, so the reconcile took 14 s (parse_count 0, stat_skipped 9893, resolve scope=Skipped).
  • The reporter's "reconciling" overview was taken inside those 14 s.
  • The longer periods they saw the day before were not timed. They will measure the next one.
  • A second restart that day was SIGTERM from the MCP client (/mcp reconnect). Its startup reconcile also took 14 s.

Item 3 (Ruby at 4.6%): probably real.

  • de.h-dv.ruby activation was re-armed at 2026-10-06T19:18:13Z.
  • The previous daemon then ran complete resolve passes between 05:35 and 05:39Z on 2026-10-07, about 70 s per stage.
  • The overview is now in steady state and still reports Ruby: 361,546 refs at 4.6%, method_call 0.5% unqualified / 5.3% qualified.
  • Caveat: nobody can prove the owner was "package" during the 05:3x passes.

Next: reproduce on a Rails app here, with builtin and package owners compared. The tier-1 corpus has no Rails app, so this is unmeasured territory for both owners.

Reporter follow-up with `daemon.log` data, 2026-10-07. This narrows the issue. **Item 1 (reconcile after restart): mostly withdrawn.** - The restart came from the idle timeout ("idle timeout reached; shutting down idle=1800s" at 06:05:15Z). - The new daemon started at 06:54:21Z. "initial reconciliation complete; watcher armed" followed at 06:54:35Z, so the reconcile took **14 s** (parse_count 0, stat_skipped 9893, resolve scope=Skipped). - The reporter's "reconciling" overview was taken inside those 14 s. - The longer periods they saw the day before were not timed. They will measure the next one. - A second restart that day was SIGTERM from the MCP client (`/mcp` reconnect). Its startup reconcile also took 14 s. **Item 3 (Ruby at 4.6%): probably real.** - `de.h-dv.ruby` activation was re-armed at 2026-10-06T19:18:13Z. - The previous daemon then ran complete resolve passes between 05:35 and 05:39Z on 2026-10-07, about 70 s per stage. - The overview is now in steady state and still reports Ruby: 361,546 refs at 4.6%, method_call 0.5% unqualified / 5.3% qualified. - Caveat: nobody can prove the owner was "package" during the 05:3x passes. **Next:** reproduce on a Rails app here, with builtin and package owners compared. The tier-1 corpus has no Rails app, so this is unmeasured territory for both owners.
Author
Member

Measured on Zammad 7.0.2: 5,155 .rb files and 359,509 Ruby refs.

The package is not the cause

The builtin and de.h-dv.ruby produce exactly the same binds, joined on path, line, col, kind and occurrence. The gap is in the host resolver.

Why refs go unresolved

A sample of 60 refs, each read at source:

Cause Refs in sample Resolvable from the source tree alone?
Rails autoload constants with no require, and Const.method calls on them 12 Yes, by structure
External gems and Ruby core/stdlib 26 No
Undeclared instance variables 15 No
Methods ActiveRecord generates from the schema 3 No
Other dynamic cases 4 No

Across the whole population, 133k unresolved refs name nothing declared in the project at all.

Fix (not yet released)

  • Constant refs now follow Ruby's own lexical lookup: enclosing scopes, then in-project ancestors and mixins, then the top level. The target is the declaration in the file that Rails' naming convention maps the constant to.
  • Const.method binds to the receiver class's def self.method. It is refused when the class also defines method=.

Results:

  • Zammad: Ruby resolution 4.58% → 11.70%; constant refs 11.7% → 64.2%.
  • Already-wrong binds: v0.33.1 had 1,240 binds pointing at the wrong constant (e.g. Ticket::Article → Sequencer::…::Ticket::Article). The new lookup corrects them, and clears 69 more phantom binds.
  • ruby-sinatra corpus repo: +116 binds and 1 phantom cleared. The operator approved re-recording its baseline rows; no other corpus repo moves.
  • Phantom audit: every new, moved and cleared bind in the samples was read at source, and all were correct.

It ships together with #327, #328 and #329. Implicit-self calls that reach concern or inherited methods remain unmeasured.

Measured on Zammad 7.0.2: 5,155 `.rb` files and 359,509 Ruby refs. ## The package is not the cause The builtin and `de.h-dv.ruby` produce exactly the same binds, joined on path, line, col, kind and occurrence. The gap is in the host resolver. ## Why refs go unresolved A sample of 60 refs, each read at source: | Cause | Refs in sample | Resolvable from the source tree alone? | |---|---|---| | Rails autoload constants with no `require`, and `Const.method` calls on them | 12 | Yes, by structure | | External gems and Ruby core/stdlib | 26 | No | | Undeclared instance variables | 15 | No | | Methods ActiveRecord generates from the schema | 3 | No | | Other dynamic cases | 4 | No | Across the whole population, 133k unresolved refs name nothing declared in the project at all. ## Fix (not yet released) - Constant refs now follow Ruby's own lexical lookup: enclosing scopes, then in-project ancestors and mixins, then the top level. The target is the declaration in the file that Rails' naming convention maps the constant to. - `Const.method` binds to the receiver class's `def self.method`. It is refused when the class also defines `method=`. Results: - **Zammad:** Ruby resolution 4.58% → 11.70%; constant refs 11.7% → 64.2%. - **Already-wrong binds:** v0.33.1 had 1,240 binds pointing at the wrong constant (e.g. `Ticket::Article` → `Sequencer::…::Ticket::Article`). The new lookup corrects them, and clears 69 more phantom binds. - **ruby-sinatra corpus repo:** +116 binds and 1 phantom cleared. The operator approved re-recording its baseline rows; no other corpus repo moves. - **Phantom audit:** every new, moved and cleared bind in the samples was read at source, and all were correct. It ships together with #327, #328 and #329. Implicit-self calls that reach concern or inherited methods remain unmeasured.
Author
Member

Released in v0.34.0 (https://git.h-dv.de/h-dv/code-index/releases/tag/v0.34.0), via PR #333.

  • Ruby's own constant lookup (12b26efd, 8129d3bb): lexical nesting first, then in-project ancestors, then the top level. Where several declarations match, the file that spells the constant wins. Const.method binds to def self.method. Wrong binds from compact class A::B::C declarations are re-pointed or cleared.
  • Measured on Zammad 7.0.2: +20,995 constants and +4,687 Const.method calls resolve, 1,884 binds moved, 69 phantom binds cleared. Ruby resolution goes 4.6% → 11.7%, constants 11.7% → 64.2%. The opt-in Ruby package is bind-for-bind identical to the builtin.
  • Migration v79 (2a2e2629) re-resolves an existing Ruby index once on upgrade. No file is re-parsed.
  • Corpus: only the ruby-sinatra baseline rows were re-recorded (operator-approved). Every other repo is byte-identical.
Released in v0.34.0 (https://git.h-dv.de/h-dv/code-index/releases/tag/v0.34.0), via PR #333. - **Ruby's own constant lookup** (12b26efd, 8129d3bb): lexical nesting first, then in-project ancestors, then the top level. Where several declarations match, the file that spells the constant wins. `Const.method` binds to `def self.method`. Wrong binds from compact `class A::B::C` declarations are re-pointed or cleared. - **Measured on Zammad 7.0.2:** +20,995 constants and +4,687 `Const.method` calls resolve, 1,884 binds moved, 69 phantom binds cleared. Ruby resolution goes 4.6% → 11.7%, constants 11.7% → 64.2%. The opt-in Ruby package is bind-for-bind identical to the builtin. - **Migration v79** (2a2e2629) re-resolves an existing Ruby index once on upgrade. No file is re-parsed. - **Corpus:** only the ruby-sinatra baseline rows were re-recorded (operator-approved). Every other repo is byte-identical.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
h-dv/code-index#326
No description provided.