h-dv/code-index

Watch

Fork 0

search_text: pinned whole-word pagination omits the generation stamp and accepts stale cursors #225

New issue

Closed

opened 2026-09-08 10:33:18 +02:00 by buildagent · 2 comments

buildagent commented

2026-09-08 10:33:18 +02:00

Member

The pinned search_text(whole_word=true) branch mints bare offsets even when the active generation is known. Consequently, it bypasses the stale-generation rejection added in #127.

Reproduction

Against this repository, compare search_text calls:

{"query":"single-writer","project":"primary","limit":1}
{"query":"single-writer","project":"primary","limit":1,"whole_word":true}

The ordinary call returns next_cursor: "1@fbe68000"; the whole-word call returns next_cursor: "1". Both have more results and both know the same generation.

In a disposable 0.27.0 index with three files containing needle, obtain both cursors, change the active generation identity, and replay them:

ordinary cursor: invalid_cursor;
whole-word cursor: accepted, another page returned with bare cursor "2".

The reproduction used controlled SQLite fault injection only in the disposable database:

UPDATE plugin_generations SET id=id+100 WHERE state='active';

This isolates cursor validation while leaving searchable text constant. It is not a test of the package activation workflow or evidence that promotion can happen mid-request. The practical stale-cursor window includes a daemon restart after promotion/rollback and snapshot-mode clients, as discussed in #127.

Cause and suggested fix

The whole-word branch constructs Some(next_off.to_string()) directly. The ordinary branch uses the shared epoch-aware next_cursor helper. parse_cursor intentionally accepts unstamped cursors for compatibility.

Replace the hand-built branch with:

let next = next_cursor(offset, limit, total_filtered, epoch.as_deref());

This retains count-only behavior and saturating arithmetic while stamping newly minted cursors whenever the generation is available. Keep acceptance of legacy unstamped cursors; the defect is minting a new unstamped cursor despite knowing the generation.

Acceptance tests

Pinned whole-word calls mint stamped cursors when an epoch is available.
Same-generation replay succeeds with the correct next page.
Changed-generation replay returns invalid_cursor.
limit=0 still returns no cursor; end-of-results and overflow handling remain correct.
Retain existing absent-epoch/older-server compatibility cases.
Test at the MCP handler level: the common helper tests already pass while this bypass remains.
Restoring the direct next_off.to_string() mint must fail the regression.

Validation

Confirmed on a freshly built 0.27.0 (a32a7519be), using MCP over stdio in disposable projects. Also dogfooded the connected 0.26.1 server against this repository. The reviewed Rust implementation is unchanged between those commits. Existing checks pass: 241 MCP binary unit tests and 8 daemon RPC integration tests; these cases are not covered by those passing tests.

Priority assessment: P2 / medium correctness. A changed result ordering can otherwise cause silent omissions or repeats. Related: #127 (closed; this is the remaining whole-word branch).

The pinned `search_text(whole_word=true)` branch mints bare offsets even when the active generation is known. Consequently, it bypasses the stale-generation rejection added in #127. ## Reproduction Against this repository, compare `search_text` calls: ```json {"query":"single-writer","project":"primary","limit":1} {"query":"single-writer","project":"primary","limit":1,"whole_word":true} ``` The ordinary call returns `next_cursor: "1@fbe68000"`; the whole-word call returns `next_cursor: "1"`. Both have more results and both know the same generation. In a disposable 0.27.0 index with three files containing `needle`, obtain both cursors, change the active generation identity, and replay them: - ordinary cursor: `invalid_cursor`; - whole-word cursor: accepted, another page returned with bare cursor `"2"`. The reproduction used controlled SQLite fault injection **only in the disposable database**: ```sql UPDATE plugin_generations SET id=id+100 WHERE state='active'; ``` This isolates cursor validation while leaving searchable text constant. It is not a test of the package activation workflow or evidence that promotion can happen mid-request. The practical stale-cursor window includes a daemon restart after promotion/rollback and snapshot-mode clients, as discussed in #127. ## Cause and suggested fix [The whole-word branch](https://git.h-dv.de/h-dv/code-index/src/commit/a32a7519beb49b22c0bbf628435cad582a576349/crates/mcp-server/src/server.rs#L16148) constructs `Some(next_off.to_string())` directly. The ordinary branch uses the shared epoch-aware `next_cursor` helper. `parse_cursor` intentionally accepts unstamped cursors for compatibility. Replace the hand-built branch with: ```rust let next = next_cursor(offset, limit, total_filtered, epoch.as_deref()); ``` This retains count-only behavior and saturating arithmetic while stamping newly minted cursors whenever the generation is available. Keep acceptance of legacy unstamped cursors; the defect is minting a new unstamped cursor despite knowing the generation. ## Acceptance tests - Pinned whole-word calls mint stamped cursors when an epoch is available. - Same-generation replay succeeds with the correct next page. - Changed-generation replay returns `invalid_cursor`. - `limit=0` still returns no cursor; end-of-results and overflow handling remain correct. - Retain existing absent-epoch/older-server compatibility cases. - Test at the MCP handler level: the common helper tests already pass while this bypass remains. - Restoring the direct `next_off.to_string()` mint must fail the regression. ## Validation Confirmed on a freshly built **0.27.0 (a32a7519beb4)**, using MCP over stdio in disposable projects. Also dogfooded the connected 0.26.1 server against this repository. The reviewed Rust implementation is unchanged between those commits. Existing checks pass: **241 MCP binary unit tests and 8 daemon RPC integration tests**; these cases are not covered by those passing tests. Priority assessment: P2 / medium correctness. A changed result ordering can otherwise cause silent omissions or repeats. Related: #127 (closed; this is the remaining whole-word branch).

buildagent added the

code-review

correctness

severity/medium

labels

2026-09-08 10:33:24 +02:00

buildagent referenced this issue

2026-09-08 10:34:04 +02:00

search_text whole_word=true treats boolean/prefix FTS expressions as literal text and silently drops matches #224

buildagent commented

2026-09-08 10:34:04 +02:00

Author

Member

Shared evidence for #224, #225 and #226:

Extract the ZIP, build the reviewed checkout with cargo build --offline -p code-index-mcp -p code-index-cli, then run python3 /path/to/repro.py from that checkout. The script uses disposable fixture projects and an isolated plugin store; the generation change is fault injection only in a disposable database.

Shared evidence for #224, #225 and #226: - [Full self-review and comparison with traditional tools](https://git.h-dv.de/attachments/4241ecf2-c7fa-4098-997a-838b9f635f4f) - [Captured 0.27.0 reproduction responses and ripgrep controls](https://git.h-dv.de/attachments/50b202ee-1313-44f3-b6e7-dd121ed857e2) - [Runnable reproduction script with instructions](https://git.h-dv.de/attachments/6e671f52-f609-42c6-b00f-03d4da77a6eb) Extract the ZIP, build the reviewed checkout with `cargo build --offline -p code-index-mcp -p code-index-cli`, then run `python3 /path/to/repro.py` from that checkout. The script uses disposable fixture projects and an isolated plugin store; the generation change is fault injection only in a disposable database.

buildagent referenced this issue

2026-09-08 10:34:04 +02:00

search_text whole_word=true loses Unicode case folding and treats non-ASCII letters as word boundaries #226

dhoyer referenced this issue from a commit

2026-09-08 13:23:49 +02:00

fix: `search_text(whole_word=true)` grades the query the daemon actually ran (#224, #225, #226)

buildagent commented

2026-09-08 16:56:17 +02:00

Author

Member

Fixed in 492c0d3, on master, shipping in v0.27.0.

Whole-word pagination minted cursors without the generation stamp, so they bypassed the stale-generation rejection that every other cursor path is subject to. Whole-word cursors are now stamped and validated like the rest.

Closing on merge.

Fixed in `492c0d3`, on `master`, shipping in v0.27.0. Whole-word pagination minted cursors without the generation stamp, so they bypassed the stale-generation rejection that every other cursor path is subject to. Whole-word cursors are now stamped and validated like the rest. Closing on merge.

buildagent closed this issue

2026-09-08 16:57:00 +02:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

h-dv/code-index#225

No description provided.

Rows
Columns