mcp-server: search_text whole_word=true has a fixed 1000-candidate cap and over-reports total #15
Labels
No labels
code-review
correctness
dos
performance
security
severity/high
severity/low
severity/medium
tech-debt
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
h-dv/code-index#15
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Severity: Medium · Category: Correctness + performance
For
whole_word=true,search_textfetches a fixed 1000-candidate cap per project at offset 0, then post-filters (crates/mcp-server/src/server.rs:608,630,662-694fan-out;:2896,2957narrow). Whole-word matches beyond the first 1000 trigram candidates are unreachable, and in the fan-out pathgrand_totalreports the raw FTS total (:657) — sototaloverstates the reachable whole-word count andnext_cursorpromises pages that never yield the missing hits.Companion perf issue:
contains_as_word(:3026-3055) allocates a fullto_ascii_lowercase()copy of each candidate file (up to the ~2 MiB read cap) per candidate → up to ~2 GiB transient allocation across a fan-out whole-word query.Fix
limitwhole-word hits or FTS exhausted), or return anapproximate: trueflag and cap the reportedtotalto what's reachable.char_indices+eq_ignore_ascii_caseover windows).Filed from a deep multi-agent code review of the workspace (v0.5.7). Severity/category per the review; file:line refs were accurate at review time — verify against current
master.Fixed in
73f5881(branchfix/ultradeep-review-findings).Reassessed on current master:
fan_out_search_textreportedgrand_total, the raw FTS trigram total). Fixed: it now accumulates the per-project whole-word-filtered count and reports that fortotal,next_cursor, and the broad-query hint. (The narrow single-project path already reported the filteredtotal_filtered, so it wasn't over-reporting — verified.)contains_as_wordallocation — was allocating a fullto_ascii_lowercase()copy of each candidate file (≤2 MiB × every fan-out candidate). Rewritten to scan byte windows witheq_ignore_ascii_casein place — behavior-identical (both sides already ASCII-folded; a window can only align to a char boundary since the needle's first byte is never a continuation byte), and all existingcontains_as_wordunit tests stay green.Test:
mcp_smoke::search_text_whole_word_total_counts_only_word_matches—todomatchesTODO(whole word) andautodone(subword) by trigram; assertswhole_word=truereturnstotal == 1, not the raw trigram total.Left as a documented known bound (separate from the over-report bug): the whole-word post-filter fetches a fixed 1000-candidate window per project, so matches ranked beyond it aren't reachable. Full pagination (loop until
limitwhole-word hits or FTS exhausted) is a larger change; happy to file it separately if you want it pursued.