A package worker deadline during a file's first extraction leaves the file empty for good, and the daemon reports ready #588

New issue

Open

opened 2026-11-14 05:58:03 +01:00 by buildagent · 0 comments

buildagent commented

2026-11-14 05:58:03 +01:00

Member

What happens

When a language package's worker exceeds its extraction deadline (30 s by default, and the time a cold worker needs to start counts against it) on a file the index has never held before, the file is stored as a parse error with no symbols, calls or imports. The pass still succeeds, the daemon reports ready, and the stored row carries the file's current size and timestamps, so later passes skip it. The file stays empty until its bytes change.

A file that already has facts is handled differently: a transient refusal keeps the previous facts and the file is extracted again on the next pass. Only the first extraction of a file turns a transient condition into a lasting one.

Seen from a client: ready, and find_callers on a function called from that file answers zero resolved callers. The only trace is parse_errors: 1 and the file's error text package refused: host.deadline_exceeded (process).

Evidence

Reproduced on the v0.41.53 candidate (integration-1114), no product edits: the fixture of typescript_daemon_imports::named_script_imports_follow_configuration_exports_and_daemon_restart, with the JavaScript worker suspended by exact PID during the cold index until the product's own deadline ended it.

resolved callers of `target` are [], expected ["src/target.ts", "src/target.ts"]
files: 4, symbols: 2, refs: 0, refs_resolved: 0, imports: 0, parse_errors: 1
path: "test/calls.test.js", error: "package refused: host.deadline_exceeded (process)"
index complete seen=4 parsed=4 … parse_errors=1 refs_resolved=0 … resolve_scope=Full
initial reconciliation complete; watcher armed

Code path (line numbers on integration-1114):

crates/indexer/src/index.rs:15706 starts one 30 s deadline for the extraction; queueing and worker start consume it (crates/plugin-supervisor/src/supervise.rs:1758).
crates/indexer/src/index.rs:15564 classifies host.deadline_exceeded, host.worker_unavailable, host.restart_backoff and host.package_quarantined as transient.
crates/indexer/src/index.rs:14525: with a previous row the facts are kept and the file is left for the next pass; with no previous row the refusal becomes ParseResult::Error.
crates/indexer/src/writer.rs:766 stores that row with the current stat and hash; crates/indexer/src/reconcile.rs:97 then skips it on every later pass.
crates/daemon/src/main.rs:1371 marks the daemon ready after any successful pass; parse errors do not hold it back.

The same path is on origin/master (v0.41.52): the files above are unchanged by the v0.41.53 train. Inspected there, not executed.

How it was found

While investigating a red test on PR #587 (package (typescript) job: the test read [] straight after ready). A side investigation reproduced the mechanism above by suspending a worker. Correction, 2026-11-14: the CI failure itself had a different cause. The same test failed again in round 3 with the new state-printing assertion (838cc557), and the daemon log shows the TypeScript package refused at start as extractor.malformed, witness extractor.tsx.wasm: the 30 s precompile budget on a slow runner, tracked in #589. The defect described here is real and reproduced, but it has not been observed in CI.

CPU starvation alone does not reproduce it: six runs on one core shared with six busy loops passed (134 s each).

Fix status

Lane codex-r46 (local, not in PR #587): a stored transient refusal is never treated as current; the shared index entry makes one more pass when a pass had transient refusals; a persistent refusal reaches ready, stays owed and is shown with "will be retried on the next pass". Replaying the reproduction against the lane: first pass parse_errors=1 refs_resolved=0, second pass parse_errors=0 refs_resolved=2, then ready.

Who is affected

Any first index on a slow or busy machine: a cold worker start plus a first file can exceed the deadline. The result is a file that is silently absent from callers, references and impact answers until it is edited.

Fix requirements

A transient refusal never leaves a row that later passes treat as current; the next pass extracts the file again.
A first index that hit transient refusals retries those files a bounded number of times before ready.
One file that always exceeds the deadline must not keep the daemon from becoming ready.
While a file is owed, the payload says so.
Rows already stored this way by earlier releases are retried too.

## What happens When a language package's worker exceeds its extraction deadline (30 s by default, and the time a cold worker needs to start counts against it) on a file the index has never held before, the file is stored as a parse error with no symbols, calls or imports. The pass still succeeds, the daemon reports `ready`, and the stored row carries the file's current size and timestamps, so later passes skip it. The file stays empty until its bytes change. A file that already has facts is handled differently: a transient refusal keeps the previous facts and the file is extracted again on the next pass. Only the first extraction of a file turns a transient condition into a lasting one. Seen from a client: `ready`, and `find_callers` on a function called from that file answers zero resolved callers. The only trace is `parse_errors: 1` and the file's error text `package refused: host.deadline_exceeded (process)`. ## Evidence Reproduced on the v0.41.53 candidate (`integration-1114`), no product edits: the fixture of `typescript_daemon_imports::named_script_imports_follow_configuration_exports_and_daemon_restart`, with the JavaScript worker suspended by exact PID during the cold index until the product's own deadline ended it. ``` resolved callers of `target` are [], expected ["src/target.ts", "src/target.ts"] files: 4, symbols: 2, refs: 0, refs_resolved: 0, imports: 0, parse_errors: 1 path: "test/calls.test.js", error: "package refused: host.deadline_exceeded (process)" index complete seen=4 parsed=4 … parse_errors=1 refs_resolved=0 … resolve_scope=Full initial reconciliation complete; watcher armed ``` Code path (line numbers on `integration-1114`): 1. `crates/indexer/src/index.rs:15706` starts one 30 s deadline for the extraction; queueing and worker start consume it (`crates/plugin-supervisor/src/supervise.rs:1758`). 2. `crates/indexer/src/index.rs:15564` classifies `host.deadline_exceeded`, `host.worker_unavailable`, `host.restart_backoff` and `host.package_quarantined` as transient. 3. `crates/indexer/src/index.rs:14525`: with a previous row the facts are kept and the file is left for the next pass; with no previous row the refusal becomes `ParseResult::Error`. 4. `crates/indexer/src/writer.rs:766` stores that row with the current stat and hash; `crates/indexer/src/reconcile.rs:97` then skips it on every later pass. 5. `crates/daemon/src/main.rs:1371` marks the daemon ready after any successful pass; parse errors do not hold it back. The same path is on `origin/master` (v0.41.52): the files above are unchanged by the v0.41.53 train. Inspected there, not executed. ## How it was found While investigating a red test on PR #587 (`package (typescript)` job: the test read `[]` straight after `ready`). A side investigation reproduced the mechanism above by suspending a worker. **Correction, 2026-11-14:** the CI failure itself had a different cause. The same test failed again in round 3 with the new state-printing assertion (`838cc557`), and the daemon log shows the TypeScript package refused at start as `extractor.malformed`, witness `extractor.tsx.wasm`: the 30 s precompile budget on a slow runner, tracked in #589. The defect described here is real and reproduced, but it has not been observed in CI. CPU starvation alone does not reproduce it: six runs on one core shared with six busy loops passed (134 s each). ## Fix status Lane `codex-r46` (local, not in PR #587): a stored transient refusal is never treated as current; the shared index entry makes one more pass when a pass had transient refusals; a persistent refusal reaches `ready`, stays owed and is shown with "will be retried on the next pass". Replaying the reproduction against the lane: first pass `parse_errors=1 refs_resolved=0`, second pass `parse_errors=0 refs_resolved=2`, then `ready`. ## Who is affected Any first index on a slow or busy machine: a cold worker start plus a first file can exceed the deadline. The result is a file that is silently absent from callers, references and impact answers until it is edited. ## Fix requirements - A transient refusal never leaves a row that later passes treat as current; the next pass extracts the file again. - A first index that hit transient refusals retries those files a bounded number of times before `ready`. - One file that always exceeds the deadline must not keep the daemon from becoming ready. - While a file is owed, the payload says so. - Rows already stored this way by earlier releases are retried too.

buildagent added the

Kind/Bug

label

2026-11-14 05:58:03 +01:00

buildagent referenced this issue

2026-11-14 07:07:51 +01:00

A slow host makes the 30 s precompile budget refuse a valid package as extractor.malformed for the daemon's whole lifetime #589

buildagent referenced this issue

2026-11-14 08:56:14 +01:00

v0.41.53: restart and update behaviour a session can rely on; link add keeps the language decision; skill command removed #587

buildagent referenced this issue

2026-11-14 12:28:46 +01:00