Semantic CLI/MCP tests conflate initial readiness with query and replacement behavior under parallel CI #416

Closed
opened 2026-10-23 02:51:50 +02:00 by buildagent · 1 comment
Member

Full local nextest CI profile (12 cores, candidate 908852b1) ran 5,039 tests: 5,033 passed, four CLI query tests hit one-shot warming_up and one daemon replacement-envelope test killed an owner before proving the initial reply was ready. A sixth failure was a candidate fixture naming the current version and is separately corrected.

The CLI failures cover content, envelope and PATH-selection semantics; they use the production ten-second one-shot deadline as an implicit debug-startup bound. The same 22-test binary passes alone. A loaded direct probe of the one-shot answer test passed in 8.73 seconds. Actual compiled CLI/MCP/daemon host, builtin and worker fingerprints were checked equal after the candidate feature-alias fix, so no remaining identity split was found in these binaries.

The replacement test only checks that its first reply lacks daemon_restarted. That also passes for warming_up, after which it kills the still-starting daemon and expects a normal replacement reply. It must first observe its expected symbol before testing a replacement. Replacement calls themselves should remain single-call assertions.

Proposed narrow fix: content-only query commands use the existing debug-only startup-wait override, while the separately classified hook-budget/deadline tests retain the actual production deadline. Prove initial daemon readiness before the deliberate kill. No production timeout changes, automatic retries, CI gate removal or new lane machinery.

Related #283/#301 (one-shot query behavior), #415 (long-startup lifecycle).

Full local nextest CI profile (12 cores, candidate 908852b1) ran 5,039 tests: 5,033 passed, four CLI query tests hit one-shot warming_up and one daemon replacement-envelope test killed an owner before proving the initial reply was ready. A sixth failure was a candidate fixture naming the current version and is separately corrected. The CLI failures cover content, envelope and PATH-selection semantics; they use the production ten-second one-shot deadline as an implicit debug-startup bound. The same 22-test binary passes alone. A loaded direct probe of the one-shot answer test passed in 8.73 seconds. Actual compiled CLI/MCP/daemon host, builtin and worker fingerprints were checked equal after the candidate feature-alias fix, so no remaining identity split was found in these binaries. The replacement test only checks that its first reply lacks daemon_restarted. That also passes for warming_up, after which it kills the still-starting daemon and expects a normal replacement reply. It must first observe its expected symbol before testing a replacement. Replacement calls themselves should remain single-call assertions. Proposed narrow fix: content-only query commands use the existing debug-only startup-wait override, while the separately classified hook-budget/deadline tests retain the actual production deadline. Prove initial daemon readiness before the deliberate kill. No production timeout changes, automatic retries, CI gate removal or new lane machinery. Related #283/#301 (one-shot query behavior), #415 (long-startup lifecycle).
Author
Member

Targeted verification now passes: all 22 CLI query tests, the replacement-envelope regression with an explicit initial-ready premise, and the 11 fingerprint/9 version-site checks. The next parallel full run exposed the same pressure in blast_radius_hook::a_public_change_reports_its_dependents: its content/session-deduplication assertion received warming_up after 52.28s including nested build/startup work. Added a narrow resource group for query_cli and blast_radius_hook in the existing ci profile: maximum two concurrent tests, each reserves four global slots. This follows the existing wasm-precompile grouping, changes no production/hook deadlines and removes no tests. Timing profile and no-retry policy remain unchanged. Full validation is still in progress; not merged or released.

Targeted verification now passes: all 22 CLI query tests, the replacement-envelope regression with an explicit initial-ready premise, and the 11 fingerprint/9 version-site checks. The next parallel full run exposed the same pressure in blast_radius_hook::a_public_change_reports_its_dependents: its content/session-deduplication assertion received warming_up after 52.28s including nested build/startup work. Added a narrow resource group for query_cli and blast_radius_hook in the existing ci profile: maximum two concurrent tests, each reserves four global slots. This follows the existing wasm-precompile grouping, changes no production/hook deadlines and removes no tests. Timing profile and no-retry policy remain unchanged. Full validation is still in progress; not merged or released.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
h-dv/code-index#416
No description provided.