h-dv/code-index

Watch

Fork 0

Two daemon-leg suites fail on a `reconciling` index and pass in isolation: one signature, two victims, so it is a missing barrier and not a flake #192

New issue

Closed

opened 2026-09-06 12:16:05 +02:00 by buildagent · 1 comment

buildagent commented

2026-09-06 12:16:05 +02:00

Member

Found by a lane that was auditing something else, and reported rather than retried — which is the only reason the shape is visible.

Measured

COSI_E2E_LEG=daemon cargo test -p code-index-mcp failed on two consecutive runs, on a DIFFERENT test each time:

run	test	what the payload said
1	`index_coverage_facts_e2e`	`coverage_reasons: ["index_reconciling"]`
2	`activation_offer_e2e`	`"state": "reconciling"`, `files: 0`

Both pass in isolation — 3/3 and 2/2 respectively.

Why this is not a flake to retry

Two different victims with one signature is the tell. A flaky test is a property of the test; a shared signature across unrelated suites is a property of the harness. Both assertions ran while the index was still reconciling, and both then described exactly that state — correctly. The product was not wrong; it was asked too early, and it said so.

Note what that means: the disclosure did its job. index_reconciling and state: "reconciling" are the honest answers to "what do you know right now", and the tests treated them as failures because they expected a steady state they never waited for.

The fix is a barrier, not a retry

A retry would make the symptom rarer and the diagnosis worse: it converts "the harness has no steady-state precondition" into "this test is occasionally slow", and the next suite to grow an early assertion inherits the same defect silently. What is needed is an explicit wait-for-steady-state before the assertion, with the wait itself able to FAIL — a barrier that gives up must say it gave up on the clock, not return and let the assertion speak for it.

This repo already owns the right idiom. Phase::settle returns how it stopped — Quiesced, Stalled or Deadline — and callers assert they never stopped on Deadline; the payload-budget suites use it precisely so that "measured a server mid-pass" cannot masquerade as a measurement. That is the shape to reuse here.

Do not fix this by asserting reconciling is acceptable in these two tests. That widens the assertion to accept the state it exists to exclude, and both suites are about what a caller SEES — a caller that gets files: 0 because the index is mid-rebuild has been told something true and useless, which is the situation the offer surface is supposed to make legible.

Scope worth checking while fixing

The two victims were found by two runs. The population is "every daemon-leg suite that asserts on index content without a steady-state precondition", and nobody has enumerated it. Treat two as a floor: grep the daemon leg for assertions that read files, coverage_reasons, offers or state without a preceding settle, and report the count. An audit that fixes two and leaves eight is the same defect with better odds.

#131 (the two settle guards), #126 (a ratchet whose failure mode was contention, closed by measuring under contention rather than by widening), and the session rule that a correctness suite cannot see a slowdown — this is its mirror: a correctness suite that measures a transient state and calls it a result.

🤖 Generated with Claude Code

https://claude.ai/code/session_01K1zj5VcFJvJt3pQxe9259K

Found by a lane that was auditing something else, and **reported rather than retried** — which is the only reason the shape is visible. ## Measured `COSI_E2E_LEG=daemon cargo test -p code-index-mcp` failed on **two consecutive runs, on a DIFFERENT test each time**: | run | test | what the payload said | |---|---|---| | 1 | `index_coverage_facts_e2e` | `coverage_reasons: ["index_reconciling"]` | | 2 | `activation_offer_e2e` | `"state": "reconciling"`, `files: 0` | **Both pass in isolation** — 3/3 and 2/2 respectively. ## Why this is not a flake to retry Two different victims with **one signature** is the tell. A flaky test is a property of the test; a shared signature across unrelated suites is a property of the **harness**. Both assertions ran while the index was still reconciling, and both then described exactly that state — correctly. The product was not wrong; it was *asked too early*, and it said so. Note what that means: the disclosure did its job. `index_reconciling` and `state: "reconciling"` are the honest answers to "what do you know right now", and the tests treated them as failures because they expected a steady state they never waited for. ## The fix is a barrier, not a retry A retry would make the symptom rarer and the diagnosis worse: it converts "the harness has no steady-state precondition" into "this test is occasionally slow", and the next suite to grow an early assertion inherits the same defect silently. What is needed is an explicit wait-for-steady-state before the assertion, with the wait itself able to FAIL — a barrier that gives up must say it gave up on the clock, not return and let the assertion speak for it. This repo already owns the right idiom. `Phase::settle` returns **how** it stopped — `Quiesced`, `Stalled` or `Deadline` — and callers assert they never stopped on `Deadline`; the payload-budget suites use it precisely so that "measured a server mid-pass" cannot masquerade as a measurement. That is the shape to reuse here. **Do not** fix this by asserting `reconciling` is acceptable in these two tests. That widens the assertion to accept the state it exists to exclude, and both suites are about what a caller SEES — a caller that gets `files: 0` because the index is mid-rebuild has been told something true and useless, which is the situation the offer surface is supposed to make legible. ## Scope worth checking while fixing The two victims were found by two runs. The population is "every daemon-leg suite that asserts on index content without a steady-state precondition", and nobody has enumerated it. Treat two as a **floor**: grep the daemon leg for assertions that read `files`, `coverage_reasons`, `offers` or `state` without a preceding settle, and report the count. An audit that fixes two and leaves eight is the same defect with better odds. ## Related #131 (the two settle guards), #126 (a ratchet whose failure mode was contention, closed by measuring under contention rather than by widening), and the session rule that a correctness suite cannot see a slowdown — this is its mirror: a correctness suite that measures a *transient* state and calls it a result. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01K1zj5VcFJvJt3pQxe9259K

buildagent referenced this issue

2026-09-06 12:48:58 +02:00

Five registries were CONFIRMED blind by mutation and four have no tracker: pool_capability, resource_grading, exec_copy and shipped_binary are still green under the mutations that should redden them #178

buildagent referenced this issue

2026-09-06 16:46:07 +02:00

search_text returned a confident total: 0 for a literal its own variant scan found in the same reply, and asserted the spellings were DISJOINT searches while behaving separator-insensitively #195

dhoyer referenced this issue from a commit

2026-09-06 19:27:53 +02:00

fix: a barrier above 31 suites, a gate that sees every assignment, and #194 refused with numbers

dhoyer referenced this issue from a commit

2026-09-06 19:27:53 +02:00

test: the barrier population is 35, and the three arrivals are named

buildagent commented

2026-09-06 19:49:58 +02:00

Author

Member

FIXED in 38ccbd2, merged as a3f2218.

The root cause was one layer above the two victims this issue names. It was not two suites needing a wait; 31 suites and 99 assertions were running unbarriered against a reconciling index, and the two that failed were simply the two whose timing exposed it. Fixing those two would have left the other 29 to fail later, on a different machine, looking like a new bug each time.

So the barrier is shared and its population is PINNED rather than described: LEG_ROUTED_USING_SHARED_INITIALIZE = 35 in harness_settle_barrier.rs, with the three arrivals since named individually (answer_provenance_e2e for #181/#182, corpus_link_e2e for #191, line_text_e2e for #183) instead of absorbed into a bumped number. A count that moves without saying who moved it is how this class hides.

Mutation run: 35 -> 34 gives exit 101, left: 35 right: 34; restored, exit 0.

Closing.

FIXED in `38ccbd2`, merged as `a3f2218`. The root cause was one layer above the two victims this issue names. It was not two suites needing a wait; **31 suites and 99 assertions were running unbarriered against a `reconciling` index**, and the two that failed were simply the two whose timing exposed it. Fixing those two would have left the other 29 to fail later, on a different machine, looking like a new bug each time. So the barrier is shared and its population is PINNED rather than described: `LEG_ROUTED_USING_SHARED_INITIALIZE = 35` in `harness_settle_barrier.rs`, with the three arrivals since named individually (`answer_provenance_e2e` for #181/#182, `corpus_link_e2e` for #191, `line_text_e2e` for #183) instead of absorbed into a bumped number. A count that moves without saying who moved it is how this class hides. Mutation run: 35 -> 34 gives exit 101, `left: 35 right: 34`; restored, exit 0. Closing.

buildagent closed this issue

2026-09-06 19:50:01 +02:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

h-dv/code-index#192

No description provided.

Rows
Columns

Two daemon-leg suites fail on a reconciling index and pass in isolation: one signature, two victims, so it is a missing barrier and not a flake #192

Measured

Why this is not a flake to retry

The fix is a barrier, not a retry

Scope worth checking while fixing

Related

Two daemon-leg suites fail on a `reconciling` index and pass in isolation: one signature, two victims, so it is a missing barrier and not a flake #192