A slow host makes the 30 s precompile budget refuse a valid package as `extractor.malformed` for the daemon's whole lifetime #589

New issue

Open

opened 2026-11-14 07:07:51 +01:00 by buildagent · 0 comments

buildagent commented

2026-11-14 07:07:51 +01:00

Member

What happens

At start, each approved package's extractors are compiled by code-index-plugin-host --precompile, one subprocess per extractor lane, each under a wall-clock budget of 30 s (PRECOMPILE_WALL_CLOCK_BUDGET_MS, crates/indexer/src/packages.rs). When a compile is still running at 30 s it is killed and the whole package is refused with reason extractor.malformed and the extractor's file name as witness (precompile_within, same file).

Two things are wrong with that on a busy machine:

The reason blames the package. extractor.malformed is also what a genuinely invalid module gets. The log line carries only reason and witness, so "this host was too slow" and "this package is broken" read the same. The refusal's bound name is not in the log line.
The budget measures the neighbours. It is wall-clock time, so load from other processes counts against the package. The package is then absent for the daemon's whole lifetime: the daemon reports ready and answers without that language's facts. The next start compiles again (the extractor cache is written only after a successful compile).

Evidence

PR #587, round 3, job package (javascript), test named_script_imports_follow_configuration_exports_and_daemon_restart (two packages, four extractor lanes, debug plugin host):

startup phase phase="package_discovery"                     05:24:33.9
WARN plugin package refused … reason="extractor.malformed" witness="extractor.tsx.wasm"   05:25:37.2
INFO plugin activation activation=1 package(s) active, 1 refused, 0 awaiting approval
index complete seen=4 parsed=4 … parse_errors=0 refs_resolved=0
stats: … refs: 5, refs_resolved: 0 … package_refusals: Some(["extractor.malformed"])

Package discovery took 63 s; the build phase of that job took 570 s against 335 s in round 1, so the runner was slow. The same test failed the same way in round 2 on the package (typescript) leg (before the assertion printed state, so that round's cause is inferred). Cargo.toml already records this failure shape from package-ci run 1041 and from a local run (observed: 30000); the mitigation then was to optimise the code generator in debug builds.

Not measured: how long the release build's precompile takes on a loaded machine. On an idle one it is far below 30 s, so this is most likely to hit debug builds and CI.

Fix requirements

A compile that ran out of time is reported as a host condition with its bound named in the log and the payload, not as extractor.malformed.
The budget bounds what the package costs (the compiler's CPU time), with a generous wall-clock backstop for a blocked process. Linux, macOS and Windows.
A package refused for a host condition is tried again without a restart, and the payload says it is owed. Same contract as #588.

## What happens At start, each approved package's extractors are compiled by `code-index-plugin-host --precompile`, one subprocess per extractor lane, each under a wall-clock budget of 30 s (`PRECOMPILE_WALL_CLOCK_BUDGET_MS`, `crates/indexer/src/packages.rs`). When a compile is still running at 30 s it is killed and the whole package is refused with reason `extractor.malformed` and the extractor's file name as witness (`precompile_within`, same file). Two things are wrong with that on a busy machine: 1. **The reason blames the package.** `extractor.malformed` is also what a genuinely invalid module gets. The log line carries only reason and witness, so "this host was too slow" and "this package is broken" read the same. The refusal's bound name is not in the log line. 2. **The budget measures the neighbours.** It is wall-clock time, so load from other processes counts against the package. The package is then absent for the daemon's whole lifetime: the daemon reports `ready` and answers without that language's facts. The next start compiles again (the extractor cache is written only after a successful compile). ## Evidence PR #587, round 3, job `package (javascript)`, test `named_script_imports_follow_configuration_exports_and_daemon_restart` (two packages, four extractor lanes, debug plugin host): ``` startup phase phase="package_discovery" 05:24:33.9 WARN plugin package refused … reason="extractor.malformed" witness="extractor.tsx.wasm" 05:25:37.2 INFO plugin activation activation=1 package(s) active, 1 refused, 0 awaiting approval index complete seen=4 parsed=4 … parse_errors=0 refs_resolved=0 stats: … refs: 5, refs_resolved: 0 … package_refusals: Some(["extractor.malformed"]) ``` Package discovery took 63 s; the build phase of that job took 570 s against 335 s in round 1, so the runner was slow. The same test failed the same way in round 2 on the `package (typescript)` leg (before the assertion printed state, so that round's cause is inferred). `Cargo.toml` already records this failure shape from package-ci run 1041 and from a local run (`observed: 30000`); the mitigation then was to optimise the code generator in debug builds. Not measured: how long the release build's precompile takes on a loaded machine. On an idle one it is far below 30 s, so this is most likely to hit debug builds and CI. ## Fix requirements - A compile that ran out of time is reported as a host condition with its bound named in the log and the payload, not as `extractor.malformed`. - The budget bounds what the package costs (the compiler's CPU time), with a generous wall-clock backstop for a blocked process. Linux, macOS and Windows. - A package refused for a host condition is tried again without a restart, and the payload says it is owed. Same contract as #588.

buildagent added the

Kind/Bug

label

2026-11-14 07:07:51 +01:00

buildagent referenced this issue

2026-11-14 07:07:52 +01:00

A package worker deadline during a file's first extraction leaves the file empty for good, and the daemon reports ready #588

buildagent referenced this issue

2026-11-14 08:56:14 +01:00

v0.41.53: restart and update behaviour a session can rely on; link add keeps the language decision; skill command removed #587

buildagent referenced this issue

2026-11-14 12:28:46 +01:00