Fix busy-daemon recovery and remove expensive Windows build-output caches #411

Merged
buildagent merged 6 commits from codex/post-upgrade-recovery into master 2026-10-23 00:43:32 +02:00
Member

During promotion after a core/package upgrade, aggregate stats requests took 17–50 seconds. Reconnecting ran another stats query, then spawned a redundant daemon while the owner continued working. Windows qualification also spent tens of minutes archiving multi-gigabyte build outputs after validation had finished.

Reconnect now uses authenticated health identity, falling back to stats only for an explicit unknown-method response. Response IDs, root mismatches, malformed replies and authentication failures are checked. Failed-reconnect recovery defers a replacement only with a held kernel lock, a live owner, no announced clean exit, and takeover's existing startup predicate or completed health probe. Silent old owners past grace and stale registrations still reach normal recovery; unknown start age retains the existing conservative policy. The original reconnect failure remains in diagnostics.

Coverage reasons emit package_id_already_active once when two sources overlap, preserving the other reasons and order. The daemon-death regression now counts actual replacement executions rather than a log message.

Windows CI and release builds cache only Cargo registry/index, registry/cache and git/db, under a new shared source-only prefix. The shared setup resolves the effective CARGO_HOME (with the normal user-profile fallback) and exports the cache path using UTF-8 without a BOM for PowerShell 5.1. Legacy target-cache fallback and the restored-build-script repair step are removed. Every non-cache step and all job-level gates are unchanged, verified by parsed YAML comparison. Cold compiled outputs build normally. This mitigates #408's archive overhead; the cache backend's reported commit failure remains a separate issue.

Local validation passed: 275 daemon unit tests, 331 MCP unit tests, 13 targeted integration tests, all 9 daemon-death tests, all 37 serialized timing tests, all 48 CI-routing tests, all 11 CI-cadence tests, and the release build-group check. Clippy and formatting passed. Required CI applies to the final head. No package bytes, schema or plugin ABI changes. Conservative upgrade rebuilding remains tracked in #410 and #412.

During promotion after a core/package upgrade, aggregate stats requests took 17–50 seconds. Reconnecting ran another stats query, then spawned a redundant daemon while the owner continued working. Windows qualification also spent tens of minutes archiving multi-gigabyte build outputs after validation had finished. Reconnect now uses authenticated health identity, falling back to stats only for an explicit unknown-method response. Response IDs, root mismatches, malformed replies and authentication failures are checked. Failed-reconnect recovery defers a replacement only with a held kernel lock, a live owner, no announced clean exit, and takeover's existing startup predicate or completed health probe. Silent old owners past grace and stale registrations still reach normal recovery; unknown start age retains the existing conservative policy. The original reconnect failure remains in diagnostics. Coverage reasons emit package_id_already_active once when two sources overlap, preserving the other reasons and order. The daemon-death regression now counts actual replacement executions rather than a log message. Windows CI and release builds cache only Cargo registry/index, registry/cache and git/db, under a new shared source-only prefix. The shared setup resolves the effective CARGO_HOME (with the normal user-profile fallback) and exports the cache path using UTF-8 without a BOM for PowerShell 5.1. Legacy target-cache fallback and the restored-build-script repair step are removed. Every non-cache step and all job-level gates are unchanged, verified by parsed YAML comparison. Cold compiled outputs build normally. This mitigates #408's archive overhead; the cache backend's reported commit failure remains a separate issue. Local validation passed: 275 daemon unit tests, 331 MCP unit tests, 13 targeted integration tests, all 9 daemon-death tests, all 37 serialized timing tests, all 48 CI-routing tests, all 11 CI-cadence tests, and the release build-group check. Clippy and formatting passed. Required CI applies to the final head. No package bytes, schema or plugin ABI changes. Conservative upgrade rebuilding remains tracked in #410 and #412.
fix: reconnect through health without respawning a busy owner
Some checks failed
Package CI / package (ruby) (pull_request) Successful in 29s
Package CI / package (xaml) (pull_request) Successful in 28s
Package CI / package (python) (pull_request) Successful in 38s
Package CI / package-corpus (python) (pull_request) Successful in 38s
Package CI / package (typescript) (pull_request) Successful in 37s
Package CI / package-corpus (timeline) (pull_request) Successful in 39s
Package CI / package (timeline) (pull_request) Successful in 47s
Package CI / package-corpus (php) (pull_request) Successful in 47s
Package CI / package-corpus (typescript) (pull_request) Successful in 46s
Package CI / package-corpus (rust) (pull_request) Successful in 57s
Package CI / package-corpus (vue) (pull_request) Successful in 53s
Package CI / package-corpus (xaml) (pull_request) Successful in 46s
Package CI / package overhead budgets (nightly, one package at a time) (pull_request) Has been skipped
CI / guest crates (fmt, clippy, doc) (pull_request) Successful in 1m0s
CI / CI lane wall-clock headroom (pull_request) Successful in 1m25s
Pilot plugin compatibility / Fetch the pinned runtime once (pull_request) Successful in 1m46s
Pilot plugin compatibility / Pilot compatibility (csharp) (pull_request) Successful in 58s
Pilot plugin compatibility / Pilot compatibility (timeline) (pull_request) Successful in 1m9s
Pilot plugin compatibility / Pilot compatibility result (pull_request) Successful in 1s
CI / cargo doc (intra-doc links) (pull_request) Successful in 4m45s
CI / cargo deny (pull_request) Successful in 5m29s
CI / cargo clippy (pull_request) Successful in 5m41s
CI / cargo check (MSRV 1.98) (pull_request) Successful in 6m31s
CI / cargo test (abi, 32-bit + wasm32) (pull_request) Successful in 6m34s
CI / cargo check (windows-gnu) (pull_request) Successful in 7m10s
CI / cargo test (daemon transport) (pull_request) Has been cancelled
CI / Plugin path cost + pool throughput (nightly) (pull_request) Has been cancelled
CI / OSS corpus (tier 1) (pull_request) Has been cancelled
CI (Windows) / fmt + clippy + build + test (windows) (pull_request) Has been cancelled
CI / cargo test (pull_request) Has been cancelled
04b19b59a4
fix: preserve takeover for silent owners and stale registrations
Some checks failed
Package CI / package (javascript) (pull_request) Successful in 38s
Package CI / package-corpus (python) (pull_request) Successful in 27s
Package CI / package (typescript) (pull_request) Successful in 45s
Package CI / package-corpus (rust) (pull_request) Successful in 37s
Package CI / package-corpus (vue) (pull_request) Successful in 35s
Package CI / package (php) (pull_request) Successful in 46s
Package CI / package-corpus (javascript) (pull_request) Successful in 46s
Package CI / package-corpus (timeline) (pull_request) Successful in 46s
Package CI / package-corpus (php) (pull_request) Successful in 52s
Package CI / package-corpus (typescript) (pull_request) Successful in 50s
Package CI / package-corpus (svelte) (pull_request) Successful in 55s
Package CI / package-corpus (xaml) (pull_request) Successful in 55s
Package CI / package overhead budgets (nightly, one package at a time) (pull_request) Has been skipped
Pilot plugin compatibility / Fetch the pinned runtime once (pull_request) Successful in 1m32s
CI / CI lane wall-clock headroom (pull_request) Successful in 1m17s
CI / guest crates (fmt, clippy, doc) (pull_request) Successful in 1m16s
Pilot plugin compatibility / Pilot compatibility (csharp) (pull_request) Successful in 1m4s
Pilot plugin compatibility / Pilot compatibility (timeline) (pull_request) Successful in 1m13s
Pilot plugin compatibility / Pilot compatibility result (pull_request) Successful in 1s
CI / cargo test (pull_request) Has been cancelled
CI / cargo test (daemon transport) (pull_request) Has been cancelled
CI / Plugin path cost + pool throughput (nightly) (pull_request) Has been cancelled
CI / cargo test (abi, 32-bit + wasm32) (pull_request) Has been cancelled
CI / cargo deny (pull_request) Has been cancelled
CI / OSS corpus (tier 1) (pull_request) Has been cancelled
CI (Windows) / fmt + clippy + build + test (windows) (pull_request) Has been cancelled
CI / cargo clippy (pull_request) Has been cancelled
CI / cargo check (MSRV 1.98) (pull_request) Has been cancelled
CI / cargo check (windows-gnu) (pull_request) Has been cancelled
CI / cargo doc (intra-doc links) (pull_request) Has been cancelled
d898641cf9
buildagent changed title from Avoid expensive reconnect handshakes and redundant busy-daemon respawns to Use health for reconnect and preserve safe daemon recovery 2026-10-22 22:42:50 +02:00
test: cover stale responsive ports and heartbeat recovery through RPC
Some checks failed
Package CI / package (xaml) (pull_request) Successful in 33s
Package CI / package (ruby) (pull_request) Successful in 37s
Package CI / package-corpus (ruby) (pull_request) Successful in 36s
Package CI / package (svelte) (pull_request) Successful in 46s
Package CI / package-corpus (svelte) (pull_request) Successful in 38s
Package CI / package-corpus (csharp) (pull_request) Successful in 50s
Package CI / package-corpus (rust) (pull_request) Successful in 54s
Package CI / package-corpus (typescript) (pull_request) Successful in 49s
Package CI / package-corpus (php) (pull_request) Successful in 46s
Package CI / package-corpus (timeline) (pull_request) Successful in 51s
Package CI / package-corpus (xaml) (pull_request) Successful in 51s
Package CI / package-corpus (vue) (pull_request) Successful in 49s
Package CI / package overhead budgets (nightly, one package at a time) (pull_request) Has been skipped
CI / guest crates (fmt, clippy, doc) (pull_request) Successful in 41s
CI / CI lane wall-clock headroom (pull_request) Successful in 1m22s
Pilot plugin compatibility / Fetch the pinned runtime once (pull_request) Successful in 1m38s
Pilot plugin compatibility / Pilot compatibility (csharp) (pull_request) Successful in 56s
Pilot plugin compatibility / Pilot compatibility (timeline) (pull_request) Successful in 1m8s
Pilot plugin compatibility / Pilot compatibility result (pull_request) Successful in 1s
CI / cargo clippy (pull_request) Successful in 4m50s
CI / cargo doc (intra-doc links) (pull_request) Successful in 5m4s
CI / cargo deny (pull_request) Successful in 6m19s
CI / cargo test (abi, 32-bit + wasm32) (pull_request) Successful in 6m38s
CI / cargo check (MSRV 1.98) (pull_request) Successful in 6m49s
CI / cargo check (windows-gnu) (pull_request) Successful in 7m2s
CI / cargo test (pull_request) Failing after 21m21s
CI / cargo test (daemon transport) (pull_request) Has been skipped
CI / OSS corpus (tier 1) (pull_request) Successful in 31m0s
CI / Plugin path cost + pool throughput (nightly) (pull_request) Has been skipped
CI (Windows) / fmt + clippy + build + test (windows) (pull_request) Has been cancelled
e1088879ec
ci: cache only Cargo sources on Windows
Some checks failed
Package CI / package (ruby) (pull_request) Successful in 31s
Package CI / package (vue) (pull_request) Successful in 33s
Package CI / package-corpus (ruby) (pull_request) Successful in 32s
Package CI / package (typescript) (pull_request) Successful in 37s
Package CI / package-corpus (python) (pull_request) Successful in 36s
Package CI / package-corpus (timeline) (pull_request) Successful in 42s
Pilot plugin compatibility / Fetch the pinned runtime once (pull_request) Successful in 1m5s
Package CI / package-corpus (csharp) (pull_request) Successful in 51s
Package CI / package-corpus (rust) (pull_request) Successful in 50s
Package CI / package-corpus (typescript) (pull_request) Successful in 46s
Package CI / package-corpus (php) (pull_request) Successful in 1m0s
Package CI / package-corpus (xaml) (pull_request) Successful in 49s
Package CI / package-corpus (vue) (pull_request) Successful in 53s
Package CI / package overhead budgets (nightly, one package at a time) (pull_request) Has been skipped
CI / guest crates (fmt, clippy, doc) (pull_request) Successful in 1m12s
CI / CI lane wall-clock headroom (pull_request) Successful in 1m25s
Pilot plugin compatibility / Pilot compatibility (csharp) (pull_request) Successful in 1m13s
Pilot plugin compatibility / Pilot compatibility (timeline) (pull_request) Successful in 1m14s
Pilot plugin compatibility / Pilot compatibility result (pull_request) Successful in 1s
CI / cargo doc (intra-doc links) (pull_request) Successful in 4m51s
CI / cargo clippy (pull_request) Successful in 5m26s
CI / cargo deny (pull_request) Successful in 6m19s
CI / cargo check (MSRV 1.98) (pull_request) Successful in 6m52s
CI / cargo test (abi, 32-bit + wasm32) (pull_request) Successful in 7m2s
CI / cargo check (windows-gnu) (pull_request) Successful in 7m22s
CI / cargo test (daemon transport) (pull_request) Has been cancelled
CI / Plugin path cost + pool throughput (nightly) (pull_request) Has been cancelled
CI / OSS corpus (tier 1) (pull_request) Has been cancelled
CI (Windows) / fmt + clippy + build + test (windows) (pull_request) Has been cancelled
CI / cargo test (pull_request) Has been cancelled
8623dd47d7
buildagent changed title from Use health for reconnect and preserve safe daemon recovery to Fix busy-daemon recovery and remove expensive Windows build-output caches 2026-10-22 23:44:35 +02:00
ci: resolve the effective Cargo home for Windows caches
All checks were successful
CI / guest crates (fmt, clippy, doc) (pull_request) Successful in 59s
Package CI / package-corpus (php) (pull_request) Successful in 23s
Package CI / package-corpus (javascript) (pull_request) Successful in 27s
Package CI / package (xaml) (pull_request) Successful in 38s
Package CI / package-corpus (ruby) (pull_request) Successful in 31s
CI / CI lane wall-clock headroom (pull_request) Successful in 1m22s
Package CI / package-corpus (svelte) (pull_request) Successful in 33s
Package CI / package-corpus (csharp) (pull_request) Successful in 51s
Pilot plugin compatibility / Fetch the pinned runtime once (pull_request) Successful in 1m34s
Package CI / package-corpus (vue) (pull_request) Successful in 22s
Package CI / package-corpus (python) (pull_request) Successful in 51s
Package CI / package-corpus (xaml) (pull_request) Successful in 26s
Package CI / package-corpus (timeline) (pull_request) Successful in 44s
Package CI / package-corpus (rust) (pull_request) Successful in 52s
Package CI / package-corpus (typescript) (pull_request) Successful in 54s
Package CI / package overhead budgets (nightly, one package at a time) (pull_request) Has been skipped
Pilot plugin compatibility / Pilot compatibility (csharp) (pull_request) Successful in 1m0s
Pilot plugin compatibility / Pilot compatibility (timeline) (pull_request) Successful in 1m19s
Pilot plugin compatibility / Pilot compatibility result (pull_request) Successful in 1s
CI / cargo doc (intra-doc links) (pull_request) Successful in 4m56s
CI / cargo clippy (pull_request) Successful in 5m51s
CI / cargo check (MSRV 1.98) (pull_request) Successful in 6m34s
CI / cargo deny (pull_request) Successful in 6m50s
CI / cargo test (abi, 32-bit + wasm32) (pull_request) Successful in 7m1s
CI / cargo check (windows-gnu) (pull_request) Successful in 7m12s
CI / cargo test (pull_request) Successful in 21m10s
CI / OSS corpus (tier 1) (pull_request) Successful in 31m31s
CI / cargo test (daemon transport) (pull_request) Successful in 9m22s
CI / Plugin path cost + pool throughput (nightly) (pull_request) Has been skipped
CI (Windows) / fmt + clippy + build + test (windows) (pull_request) Successful in 39m41s
feb95a0cab
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
h-dv/code-index!411
No description provided.