perf/hang: daemon resolve stage wedges on large cross-project C# repos (tier-3 import-boost GLOB cross-product) #65

Open
opened 2026-08-05 12:37:51 +02:00 by buildagent · 2 comments
Member

Source: a1x dogfood on a 235-project .NET solution (~9k source files after build-output exclusion). Extreme instance of #53 (tier-3 scaling): here it does not terminate on human timescales. Release-blocker for that class of repo — not yet fixed.

Symptom

Fresh index: ~70s of real indexing (index.db 0 → 713 MB), then the daemon pegs ONE core with ZERO further I/O — index.db AND index.db-wal mtimes frozen — indefinitely (>10 min observed; 4 of 5 runs). Thread dump: 18 threads, exactly one Running (all process CPU), flat ~173 MB WS → spinning, not allocating. Last log before the freeze: outline_changed=8958 at index stage: resolving references. The RPC listener stays up (TCP connects) but every tool returns warming_up.

Bisect (reporter)

Subtrees resolve fine and fast; only the full 235-project root wedges. tier3_groups: 0 on a single project, 297 on a mid subtree, and it explodes at full cross-project import fan-out. The wedge scales with tier-3 group count.

Root cause (source analysis)

tier-3 import-boost resolution in resolve_ref_targets_in_tx (crates/indexer/src/index.rs, the ambiguous-remain block) joins each still-ambiguous ref's candidate pool against temp.file_keys with NON-INDEXABLE leading-* GLOBs, e.g. {IMPORT_SEGMENTS} GLOB ('*.' || k.key || '.*') (~lines 1408-1435), plus the reachability predicate. A leading-* GLOB can't use an index → full scan each time → evaluated across {ambiguous refs} × {candidate symbols} × {file_keys}. At 235-project fan-out this is super-linear and effectively non-terminating. Verified there is NO recursive CTE / worklist loop — it terminates in theory, just not in practice. Same class as I020 (Cap'n Proto quadratic); the de-correlation note at index.rs:1478 isn't enough at this scale.

Suggested direction

  1. Make tier-3 reachability indexable — materialize a per-file import-reachable set and join on equality instead of correlated leading-* GLOBs.
  2. Budget tier-3: when group-count × mean-pool exceeds a cost bound, DEGRADE those refs to name_fallback with disclosure instead of scanning the cross-product.
  3. A hard wall-clock/row budget on the resolve stage so a pathological repo degrades-with-disclosure ("gave up on N refs") rather than wedging. (Pairs with the readiness/observability gaps — see the daemon-lifecycle issue.)

Secondary (same repo) — index size

~115 KB of index per source file (~9k files → 1.04 GB) with build output EXCLUDED — not generated JSON. Candidate: per-occurrence ref rows carrying qualified-name strings (52,989 refs / 114 files in one subtree ⇒ tens of millions of rows at full scale). Worth checking whether ref storage is the real size driver. May share a cause with the tier-3 working-set blowup.

Related: #53.

**Source:** a1x dogfood on a 235-project .NET solution (~9k source files after build-output exclusion). Extreme instance of #53 (tier-3 scaling): here it does not terminate on human timescales. **Release-blocker for that class of repo — not yet fixed.** ## Symptom Fresh index: ~70s of real indexing (index.db 0 → 713 MB), then the daemon pegs ONE core with ZERO further I/O — index.db AND index.db-wal mtimes frozen — indefinitely (>10 min observed; 4 of 5 runs). Thread dump: 18 threads, exactly one Running (all process CPU), flat ~173 MB WS → spinning, not allocating. Last log before the freeze: `outline_changed=8958` at `index stage: resolving references`. The RPC listener stays up (TCP connects) but every tool returns `warming_up`. ## Bisect (reporter) Subtrees resolve fine and fast; only the full 235-project root wedges. `tier3_groups`: 0 on a single project, 297 on a mid subtree, and it explodes at full cross-project import fan-out. The wedge scales with tier-3 group count. ## Root cause (source analysis) tier-3 import-boost resolution in `resolve_ref_targets_in_tx` (crates/indexer/src/index.rs, the ambiguous-remain block) joins each still-ambiguous ref's candidate pool against `temp.file_keys` with NON-INDEXABLE leading-`*` GLOBs, e.g. `{IMPORT_SEGMENTS} GLOB ('*.' || k.key || '.*')` (~lines 1408-1435), plus the reachability predicate. A leading-`*` GLOB can't use an index → full scan each time → evaluated across {ambiguous refs} × {candidate symbols} × {file_keys}. At 235-project fan-out this is super-linear and effectively non-terminating. Verified there is NO recursive CTE / worklist loop — it terminates in theory, just not in practice. Same class as I020 (Cap'n Proto quadratic); the de-correlation note at index.rs:1478 isn't enough at this scale. ## Suggested direction 1. Make tier-3 reachability indexable — materialize a per-file import-reachable set and join on equality instead of correlated leading-`*` GLOBs. 2. Budget tier-3: when group-count × mean-pool exceeds a cost bound, DEGRADE those refs to `name_fallback` with disclosure instead of scanning the cross-product. 3. A hard wall-clock/row budget on the resolve stage so a pathological repo degrades-with-disclosure ("gave up on N refs") rather than wedging. (Pairs with the readiness/observability gaps — see the daemon-lifecycle issue.) ## Secondary (same repo) — index size ~115 KB of index per source file (~9k files → 1.04 GB) with build output EXCLUDED — not generated JSON. Candidate: per-occurrence ref rows carrying qualified-name strings (52,989 refs / 114 files in one subtree ⇒ tens of millions of rows at full scale). Worth checking whether ref storage is the real size driver. May share a cause with the tier-3 working-set blowup. Related: #53.
Author
Member

Status update — NOT closing. Suggested direction 3 is now shipped; the secondary remains open.

Landed today on master (a5adb00), completing suggested direction 3 ("a hard budget so a pathological repo degrades-with-disclosure rather than wedging"):

The tier-3 budget degradation already existed, but its only outputs were a tracing::warn! and a #[cfg(test)] atomic — a workspace grep found no consumer outside index.rs. So an entire resolution tier could be skipped and no MCP tool could report it: affected refs looked like ordinary misses and every published resolution rate was quietly wrong. The "with disclosure" half of the degrade was missing.

Now:

  • persisted in a new resolver_health table (m0032, schema v32), re-derived on every resolve — written on both branches, so a set-only flag cannot latch on after one degraded run;
  • surfaced as project_overview.resolver_degradation and resolution_gaps.resolver_degradation, carrying the flag, the budget in force (so it is actionable, not merely announced), and a semantics string stating the consequence — for resolution_gaps specifically, that an unknown share of the counts below are budget artifacts, not resolver blind spots, and the table should not be worked as a recall backlog until the budget is raised;
  • three-state: absent means "this index did not report" (pre-m0032 database) and is deliberately not Some(false). Mutation-proven — collapsing that with unwrap_or(false) fails the test, which matters because it looks like harmless defensive coding and would tell every older index "no degradation occurred".

Directions 1 (indexable tier-3 reachability) and 2 (budget → name_fallback) were addressed by the v0.10.0 work; the wedge itself is fixed.

Still open: the Secondary section — ~115 KB of index per source file (~9k files → 1.04 GB) with build output excluded, and the hypothesis that per-occurrence ref rows carrying qualified-name strings are the size driver. Nothing in this session touched storage size. That question deserves its own issue if you'd rather close this one; leaving it here for now so the measurement isn't lost.

**Status update — NOT closing. Suggested direction 3 is now shipped; the secondary remains open.** Landed today on `master` (`a5adb00`), completing suggested direction **3** ("a hard budget so a pathological repo degrades-with-disclosure rather than wedging"): The tier-3 budget degradation already existed, but its *only* outputs were a `tracing::warn!` and a `#[cfg(test)]` atomic — a workspace grep found no consumer outside `index.rs`. So an entire resolution tier could be skipped and **no MCP tool could report it**: affected refs looked like ordinary misses and every published resolution rate was quietly wrong. The "with disclosure" half of the degrade was missing. Now: - persisted in a new `resolver_health` table (m0032, schema v32), re-derived on **every** resolve — written on both branches, so a set-only flag cannot latch on after one degraded run; - surfaced as `project_overview.resolver_degradation` **and** `resolution_gaps.resolver_degradation`, carrying the flag, the budget in force (so it is actionable, not merely announced), and a semantics string stating the consequence — for `resolution_gaps` specifically, that an unknown share of the counts below are **budget artifacts, not resolver blind spots**, and the table should not be worked as a recall backlog until the budget is raised; - three-state: absent means *"this index did not report"* (pre-m0032 database) and is deliberately **not** `Some(false)`. Mutation-proven — collapsing that with `unwrap_or(false)` fails the test, which matters because it looks like harmless defensive coding and would tell every older index "no degradation occurred". Directions 1 (indexable tier-3 reachability) and 2 (budget → `name_fallback`) were addressed by the v0.10.0 work; the wedge itself is fixed. **Still open:** the *Secondary* section — ~115 KB of index per source file (~9k files → 1.04 GB) with build output excluded, and the hypothesis that per-occurrence ref rows carrying qualified-name strings are the size driver. Nothing in this session touched storage size. That question deserves its own issue if you'd rather close this one; leaving it here for now so the measurement isn't lost.
Author
Member

Secondary (index size) — measured. The hypothesis in the issue body is wrong.

The body proposes "per-occurrence ref rows carrying qualified-name strings" as the size driver. Measured with dbstat on two indexes:

component cosi-mcp (194 code files, 48.3 MB) py-django (2970 code files, 147.6 MB)
FTS5 total 80.9% 57.5%
files_fts_data (trigram index) 70.9% 38.4%
files_fts_content (stored doc copy) 10.0% 18.9%
refs + its indexes 13.7% 29.4%
symbols + its indexes ~3.6% ~7%

Ref rows are the second driver, not the first. Full-text search is 57–81% of the index in both.

And the obvious FTS lever was already evaluated and rejected. files_fts_content holds a full document copy (27.2 MB on django, avg 6.4 KB/file), which looks like waste until you read _prdoc/records/I001-A001-fts5-contentless.md"FTS5 contentless mode is operationally untenable". Contentless FTS5 cannot delete or replace a single row (only a full delete-all), which breaks per-file incremental updates, and search_text's snippet(files_fts, 1, …) (local_index.rs:1782) needs the stored content to produce excerpts. So that ~19% is a deliberate, documented trade-off, not a regression.

On the reported 115 KB/source-file: django measures ~34.9 KB per indexed file (147.6 MB / 4235). The C# case is therefore ~3.3× heavier per file than django, which suggests something specific to that repo or to the C# plugin rather than a general per-ref cost — worth isolating before optimising anything. Note also that FTS covers text/metadata files too (django: 4235 FTS rows vs 2970 code files), by design, so file count alone understates FTS scope.

Where the remaining levers actually are, in rough order of size: the trigram tokenizer (files_fts_data, the single largest component — trigrams of every document are inherently bulky; a cheaper tokenizer or a size cap on FTS'd files would move real bytes); then ref-row width, which is the issue's original hypothesis and is worth ~29% at django scale rather than the majority.

Recommend re-titling this secondary around FTS storage cost, since that is where the bytes are, and carrying forward the constraint that contentless mode is off the table for the reasons in I001-A001.

**Secondary (index size) — measured. The hypothesis in the issue body is wrong.** The body proposes *"per-occurrence ref rows carrying qualified-name strings"* as the size driver. Measured with `dbstat` on two indexes: | component | cosi-mcp (194 code files, 48.3 MB) | py-django (2970 code files, 147.6 MB) | |---|---|---| | **FTS5 total** | **80.9%** | **57.5%** | | ├ `files_fts_data` (trigram index) | 70.9% | 38.4% | | └ `files_fts_content` (stored doc copy) | 10.0% | 18.9% | | **refs + its indexes** | **13.7%** | **29.4%** | | `symbols` + its indexes | ~3.6% | ~7% | Ref rows are the *second* driver, not the first. Full-text search is 57–81% of the index in both. **And the obvious FTS lever was already evaluated and rejected.** `files_fts_content` holds a full document copy (27.2 MB on django, avg 6.4 KB/file), which looks like waste until you read `_prdoc/records/I001-A001-fts5-contentless.md` — *"FTS5 contentless mode is operationally untenable"*. Contentless FTS5 cannot delete or replace a single row (only a full `delete-all`), which breaks per-file incremental updates, and `search_text`'s `snippet(files_fts, 1, …)` (`local_index.rs:1782`) needs the stored content to produce excerpts. So that ~19% is a deliberate, documented trade-off, not a regression. **On the reported 115 KB/source-file:** django measures ~34.9 KB per indexed file (147.6 MB / 4235). The C# case is therefore ~3.3× heavier per file than django, which suggests something specific to that repo or to the C# plugin rather than a general per-ref cost — worth isolating before optimising anything. Note also that FTS covers text/metadata files too (django: 4235 FTS rows vs 2970 code files), by design, so file count alone understates FTS scope. **Where the remaining levers actually are**, in rough order of size: the trigram tokenizer (`files_fts_data`, the single largest component — trigrams of every document are inherently bulky; a cheaper tokenizer or a size cap on FTS'd files would move real bytes); then ref-row width, which is the issue's original hypothesis and is worth ~29% at django scale rather than the majority. Recommend re-titling this secondary around **FTS storage cost**, since that is where the bytes are, and carrying forward the constraint that contentless mode is off the table for the reasons in I001-A001.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
h-dv/code-index#65
No description provided.