Reference resolution is one non-resumable transaction, so one-shot queries after an upgrade can never converge #301

New issue

Open

opened 2026-09-26 01:12:43 +02:00 by buildagent · 2 comments

buildagent commented

2026-09-26 01:12:43 +02:00

Member

After a schema upgrade that invalidates code caches (m0070, m0071), a code-index query with no daemon running never finishes reconciling. Each call answers warming_up, and then the server is killed.

What lane H measured (I075, on a django copy upgraded from schema 69, killing code-index index every 15 s):

File reparse IS durable across kills. Files awaiting reparse went 2,970 → 1,570 → 370 → 0 over three windows.
Reference resolution is NOT durable. It runs as a single transaction (about 32 s on django), and each kill discards it. Twelve consecutive 15 s windows never completed it.
A one-shot call's window is about 2 s, and the transaction grows with repository size. Per-call progress therefore cannot converge unless the resolver itself becomes resumable.

What v0.32.1 ships: disclosure only. A one-shot warming_up carries reconcile_pending { converges_on_retry: false, … } and names the remedy: run code-index index once, or start the daemon. This is honest, but hooks such as the blast-radius hook still get no answer until a user acts.

Proposed fix: make resolution resumable. Commit per batch or per file group, with a generation-safe cursor, so each short-lived process makes durable progress. This must keep the generation-gating invariants (generation_build, generation_isolation) and the reader-epoch guarantees. A partially resolved generation must never be visible as complete.

🤖 Generated with Claude Code

After a schema upgrade that invalidates code caches (m0070, m0071), a `code-index query` with no daemon running never finishes reconciling. Each call answers `warming_up`, and then the server is killed. **What lane H measured** (I075, on a django copy upgraded from schema 69, killing `code-index index` every 15 s): - File reparse IS durable across kills. Files awaiting reparse went 2,970 → 1,570 → 370 → 0 over three windows. - Reference resolution is NOT durable. It runs as a single transaction (about 32 s on django), and each kill discards it. Twelve consecutive 15 s windows never completed it. - A one-shot call's window is about 2 s, and the transaction grows with repository size. Per-call progress therefore cannot converge unless the resolver itself becomes resumable. **What v0.32.1 ships:** disclosure only. A one-shot `warming_up` carries `reconcile_pending { converges_on_retry: false, … }` and names the remedy: run `code-index index` once, or start the daemon. This is honest, but hooks such as the blast-radius hook still get no answer until a user acts. **Proposed fix:** make resolution resumable. Commit per batch or per file group, with a generation-safe cursor, so each short-lived process makes durable progress. This must keep the generation-gating invariants (`generation_build`, `generation_isolation`) and the reader-epoch guarantees. A partially resolved generation must never be visible as complete. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

buildagent referenced this issue

2026-09-26 03:27:38 +02:00

release v0.32.1 — fix the v0.32.0 review findings #302

buildagent commented

2026-09-26 22:26:40 +02:00

Author

Member

Still current as of 0bcdac1 (checked by Phase 1 lane R on 2026-09-26): resolution is one transaction in apply_resolution_observed, so a one-shot code-index query answers warming_up with converges_on_retry: false.

Two options:

Full fix: batch the roughly 6,000-line Full resolver by file, with a durable cursor and pools rebuilt per batch. This puts baseline equivalence and the cost ratchet at risk.
Cheaper: a one-shot query that finds a pending reconcile and no daemon starts a detached code-index index to finish it, and says so in its reply. Retries then converge without a daemon.

This is a prerequisite before any language flips to a package in Phase 3.

Still current as of 0bcdac1 (checked by Phase 1 lane R on 2026-09-26): resolution is one transaction in `apply_resolution_observed`, so a one-shot `code-index query` answers `warming_up` with `converges_on_retry: false`. Two options: - **Full fix:** batch the roughly 6,000-line Full resolver by file, with a durable cursor and pools rebuilt per batch. This puts baseline equivalence and the cost ratchet at risk. - **Cheaper:** a one-shot query that finds a pending reconcile and no daemon starts a detached `code-index index` to finish it, and says so in its reply. Retries then converge without a daemon. This is a prerequisite before any language flips to a package in Phase 3.

buildagent referenced this issue

2026-09-27 02:35:53 +02:00

Phase 1 of pluggable languages: plain ids, owner setting, opt-in enablement, ABI and profiles, cost fixes #308

buildagent commented

2026-09-27 13:45:56 +02:00

Author

Member

The nightly cargo test on master (run 5643, master at 0bcdac1) failed in query_cli::the_cli_log_level_reaches_the_spawned_server. It hit this issue directly: the one-shot code-index query answered warming_up ("the index is still reconciling and retrying this query will not finish it"), so the test never reached the premise it asserts. The same test passed in every PR run of this code and in a local full run, so it is timing-dependent: it fails when the runner is loaded, and the nightly runs heavier jobs alongside it.

Until this issue is fixed, any test that runs a one-shot query right after indexing can flake like this. Fixing this issue removes the flake class; a separate test-side workaround would only hide it.

The nightly `cargo test` on master (run 5643, master at 0bcdac1) failed in `query_cli::the_cli_log_level_reaches_the_spawned_server`. It hit this issue directly: the one-shot `code-index query` answered `warming_up` ("the index is still reconciling and retrying this query will not finish it"), so the test never reached the premise it asserts. The same test passed in every PR run of this code and in a local full run, so it is timing-dependent: it fails when the runner is loaded, and the nightly runs heavier jobs alongside it. Until this issue is fixed, any test that runs a one-shot query right after indexing can flake like this. Fixing this issue removes the flake class; a separate test-side workaround would only hide it.

buildagent referenced this issue

2026-09-27 15:03:26 +02:00

epic: every builtin language becomes a first-party plugin package #309

buildagent added a new dependency

2026-09-27 15:03:32 +02:00

#309 epic: every builtin language becomes a first-party plugin package