CI transport soak rejects a fully successful stream by aggregate throughput; allocator sweeps repeat four times #456

Closed
opened 2026-10-24 19:12:32 +02:00 by buildagent · 1 comment
Member

XAML PR #451 transport run6601/job48918 completed all 300 mixed MCP requests with matching response IDs and no protocol/tool/request-timeout error, then failed mcp_smoke.rs:6275 solely because their combined time was 45.847544073s against a 30s aggregate assertion. The harness separately bounds each response at 30s. A completed stream did not deadlock; the aggregate throughput assertion mislabels CPU contention as a correctness failure. Similar timing misattribution: #253; startup semantics are separately addressed by #416.

The same binary also runs four identical full context-pack budget sweeps over the same fixture in separate servers. The four invariants and anti-vacuity checks are valuable, but executing the identical wire requests four times adds cost and contention. A local daemon-leg baseline for the five allocation tests passes in132.27s; the CI sweeps took about23minutes while other builds/tests ran.

Candidate: retain all 300 response/protocol/tool assertions and the existing per-response 30s deadlines; report total elapsed time as a diagnostic. Capture one immutable real MCP budget sweep per test binary/transport leg with lazy initialization, retaining every budget and all four independently runnable invariant tests. No production timeout or performance-budget threshold is raised; no semantic test or budget is removed. Before/after measurement and normal CI on the corrected combined source will qualify publication.

XAML PR #451 transport run6601/job48918 completed all 300 mixed MCP requests with matching response IDs and no protocol/tool/request-timeout error, then failed mcp_smoke.rs:6275 solely because their combined time was 45.847544073s against a 30s aggregate assertion. The harness separately bounds each response at 30s. A completed stream did not deadlock; the aggregate throughput assertion mislabels CPU contention as a correctness failure. Similar timing misattribution: #253; startup semantics are separately addressed by #416. The same binary also runs four identical full context-pack budget sweeps over the same fixture in separate servers. The four invariants and anti-vacuity checks are valuable, but executing the identical wire requests four times adds cost and contention. A local daemon-leg baseline for the five allocation tests passes in132.27s; the CI sweeps took about23minutes while other builds/tests ran. Candidate: retain all 300 response/protocol/tool assertions and the existing per-response 30s deadlines; report total elapsed time as a diagnostic. Capture one immutable real MCP budget sweep per test binary/transport leg with lazy initialization, retaining every budget and all four independently runnable invariant tests. No production timeout or performance-budget threshold is raised; no semantic test or budget is removed. Before/after measurement and normal CI on the corrected combined source will qualify publication.
Author
Member

Correction committed in combined PR #455, head b7c34798d4777175ae826bdb8e569befbab4c8f6. No production code or timeout changed. Corrected soak passes all300 requests (7.62755s stream locally); the existing intentional hung-child deadline control still passes. Shared sweep captures479 real replies once per binary/transport: all5 allocation tests pass on daemon and snapshot legs, and a separately filtered invariant independently captures479 replies and passes. Every assertion/anti-vacuity/budget is retained. Local daemon-leg batch132.27s before vs116.26s after; measured snapshot batch68.60s, filtered daemon invariant130.47s (different workloads, not a latency comparison). Strict MCP test Clippy and fmt pass. Full normal CI now qualifies the corrected combined tree; no stale red or superseded running head supplies release evidence.

Correction committed in combined PR #455, head `b7c34798d4777175ae826bdb8e569befbab4c8f6`. No production code or timeout changed. Corrected soak passes all300 requests (7.62755s stream locally); the existing intentional hung-child deadline control still passes. Shared sweep captures479 real replies once per binary/transport: all5 allocation tests pass on daemon and snapshot legs, and a separately filtered invariant independently captures479 replies and passes. Every assertion/anti-vacuity/budget is retained. Local daemon-leg batch132.27s before vs116.26s after; measured snapshot batch68.60s, filtered daemon invariant130.47s (different workloads, not a latency comparison). Strict MCP test Clippy and fmt pass. Full normal CI now qualifies the corrected combined tree; no stale red or superseded running head supplies release evidence.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
h-dv/code-index#456
No description provided.