h-dv/code-index

Fork 0

feat: context_pack — task-specific, evidence-linked context under a token budget #35

New issue

Open

opened 2026-07-20 22:45:36 +02:00 by buildagent · 2 comments

buildagent commented

2026-07-20 22:45:36 +02:00

Member

Product

A task-oriented successor/complement to repo_map: given symbols, files, a diff, or a short task focus, return the smallest evidence-linked context package needed to reason or edit safely.

exact target definitions and containing types
important callers/callees, ranked by path relevance rather than global popularity
shortest dependency witness paths (#26)
affected public boundaries and tests (#22)
nearby analogous/duplicate implementations (#27)
architectural constraints/cycles (#28)
confidence gaps and suggested fallback searches
source spans as handles; bytes included only while budget remains

Why

repo_map is excellent orientation, but an edit usually triggers several sequential tool calls: search target, inspect outline, walk dependencies, find tests, search analogues, then manually fit everything into context. A server-side pack can share graph traversal, deduplicate spans, and spend tokens on evidence rather than repeated envelopes.

This is not “ask the server to understand arbitrary prose.” Initial focus inputs can be exact symbol names/IDs, paths, and changed-symbol handles. Natural-language extraction may remain client-side.

Selection algorithm

seed from resolved handles; report every missing/ambiguous seed
personalized graph ranking biased toward seeds and boundary/test nodes
reserve budget classes (targets, witnesses, tests, analogues) so one dense file cannot consume all tokens
deduplicate overlapping/containing spans
return an inclusion reason and score for every item
provide deterministic output and explicit omitted counts

Acceptance

strict token budget with measured, deterministic accounting
no silent seed misses or project-routing ambiguity
every included item states why it matters
handles can be passed directly to existing drill-down tools
benchmark against the equivalent multi-tool workflow for response size and round trips
fixtures cover multi-project links, dense generated files, ambiguous names, and a diff seed

## Product A task-oriented successor/complement to `repo_map`: given symbols, files, a diff, or a short task focus, return the smallest evidence-linked context package needed to reason or edit safely. ### Contents - exact target definitions and containing types - important callers/callees, ranked by path relevance rather than global popularity - shortest dependency witness paths (#26) - affected public boundaries and tests (#22) - nearby analogous/duplicate implementations (#27) - architectural constraints/cycles (#28) - confidence gaps and suggested fallback searches - source spans as handles; bytes included only while budget remains ## Why `repo_map` is excellent orientation, but an edit usually triggers several sequential tool calls: search target, inspect outline, walk dependencies, find tests, search analogues, then manually fit everything into context. A server-side pack can share graph traversal, deduplicate spans, and spend tokens on evidence rather than repeated envelopes. This is not “ask the server to understand arbitrary prose.” Initial focus inputs can be exact symbol names/IDs, paths, and changed-symbol handles. Natural-language extraction may remain client-side. ## Selection algorithm - seed from resolved handles; report every missing/ambiguous seed - personalized graph ranking biased toward seeds and boundary/test nodes - reserve budget classes (targets, witnesses, tests, analogues) so one dense file cannot consume all tokens - deduplicate overlapping/containing spans - return an inclusion reason and score for every item - provide deterministic output and explicit omitted counts ## Acceptance - strict token budget with measured, deterministic accounting - no silent seed misses or project-routing ambiguity - every included item states why it matters - handles can be passed directly to existing drill-down tools - benchmark against the equivalent multi-tool workflow for response size and round trips - fixtures cover multi-project links, dense generated files, ambiguous names, and a diff seed

buildagent referenced this issue

2026-07-21 14:26:17 +02:00

feat: stable symbol handles across reindexing and agent context compaction #37

buildagent referenced this issue

2026-07-21 14:26:47 +02:00

feat: duplication family — find_duplicates + drift_clones + find_similar #27

buildagent referenced this issue

2026-07-21 14:26:47 +02:00

feat: detect_cycles + check_layering — file/crate SCCs and cycle-aware edit checks #28

buildagent referenced this issue

2026-07-21 14:26:47 +02:00

feat: quick wins umbrella — 7 single-query capabilities on existing data #30

buildagent commented

2026-07-21 14:26:47 +02:00

Author

Member

Triage: P0 — next product milestone

Make this the task-oriented context compiler rather than introducing a competing task_context tool. Add an optional short task / focus_text input, used only for deterministic lexical symbol/text seed discovery; do not claim arbitrary natural-language understanding.

V1 should ship without waiting for #27 or #28 and include:

exact, missing, and ambiguous seeds
target definitions and containing symbols
important callers/callees
affected tests from change_impact
witness paths from explain_dependency
relevant imports and text/config evidence
reserved token classes, overlap deduplication, inclusion reasons, omitted counts
source bytes only while budget remains
confidence gaps and suggested drill-down

Stable handles (#37) are the cross-reindex contract and should land first or in the same milestone. Benchmark against representative coding tasks: tool calls, source tokens read, time to correct file, relevant-context precision/recall, and test-selection quality.

### Triage: P0 — next product milestone Make this the task-oriented context compiler rather than introducing a competing `task_context` tool. Add an optional short `task` / `focus_text` input, used only for deterministic lexical symbol/text seed discovery; do not claim arbitrary natural-language understanding. V1 should ship without waiting for #27 or #28 and include: - exact, missing, and ambiguous seeds - target definitions and containing symbols - important callers/callees - affected tests from `change_impact` - witness paths from `explain_dependency` - relevant imports and text/config evidence - reserved token classes, overlap deduplication, inclusion reasons, omitted counts - source bytes only while budget remains - confidence gaps and suggested drill-down Stable handles (#37) are the cross-reindex contract and should land first or in the same milestone. Benchmark against representative coding tasks: tool calls, source tokens read, time to correct file, relevant-context precision/recall, and test-selection quality.

buildagent referenced this issue

2026-07-21 14:26:51 +02:00

feat: stable symbol handles across reindexing and agent context compaction #37

buildagent added the

performance

label

2026-07-21 14:27:27 +02:00

buildagent referenced this issue from a commit

2026-07-21 15:30:55 +02:00

docs(prdoc): file mission I031 — stable handles (#37) + context_pack (#35), fixed design

buildagent referenced this issue from a commit

2026-07-21 15:30:55 +02:00

feat(mcp-server): I031 #35 — minimal context_pack (20 tools)

buildagent referenced this issue from a commit

2026-07-21 15:30:55 +02:00

fix(mcp-server): I031 #35 review round — single-project seeds, dedup, honest budget

buildagent commented

2026-07-21 15:43:30 +02:00

Author

Member

Minimal context_pack v1 shipped in v0.7.0 (mission I031, built on #37 stable handles). Keeping this issue open for the v2 vision (analogue expansion #27, cycle awareness #28, richer ranked-witness variety).

What landed (v1)

context_pack({ symbol_ids?, handles?, paths?, diff_base?, task_terms?, token_budget=4000, project }) → a single deterministic, budget-classed context bundle assembled from explicit seeds:

Single-project discipline. The pack is pinned to one project. Any seed that resolves to a different project is reported as a cross_project_seed issue rather than silently mixed in — this closes the same cross-project identity-confusion class #37 exists to prevent.
Every seed miss is reported, never silent — seed_issues carries missing / ambiguous / cross-project / no-match-for-task-term, each with its offending input.
Seeds: symbol_ids (generation-local), handles (durable, resolved via #37), paths (file → symbols, capped PATH_SYMBOL_CAP=64), diff_base (changed symbols), and task_terms — lexical focus terms each run through search_symbols; matches become seeds, no-match → a seed issue (NL extraction stays client-side).
Budget classes: targets are always kept; dependents are ranked by path-relevance and trimmed to fit; witness paths and test-role files included. One dense file can't consume the whole budget.
Honest accounting: tokens_used is the true serialized size of the response (serde_json::to_string(&resp).len()/4), and a budget_exceeded flag is set rather than silently truncating past the ceiling. MAX_SEEDS=128. Every returned item carries an inclusion reason, score, and (for symbols) a stable_handle. Dependents are deduped across chunks.

Benchmark

Against the current multi-tool workflow (self-index): context assembly is 1 pack call vs a 5-call manual sequence (search → get_symbol → find_callers → dependents → tests). The pack front-loads all evidence — targets + ~40 dependents + ~21 test-role files — into one larger payload: a clear round-trip win, with an honest byte trade-off surfaced via budget_exceeded and true-size tokens_used.

Review

Deep adversarial review (17 agents: 11 confirmed / 2 refuted), all confirmed fixed with regression tests. Headline: cross-project seed-id mixing (project-local numeric ids from different projects) — now single-project-disciplined with cross_project_seed reporting.

Deferred to v2 (this issue stays open)

Analogue expansion (#27) — pull in structurally-similar exemplars beyond direct dependents.
Cycle-aware neighborhood (#28).
Richer personalized-rank witness variety and span-dedup across overlapping ranges.

Minimal **context_pack v1** shipped in **v0.7.0** (mission I031, built on #37 stable handles). Keeping this issue **open** for the v2 vision (analogue expansion #27, cycle awareness #28, richer ranked-witness variety). ## What landed (v1) `context_pack({ symbol_ids?, handles?, paths?, diff_base?, task_terms?, token_budget=4000, project })` → a single deterministic, budget-classed context bundle assembled from explicit seeds: - **Single-project discipline.** The pack is pinned to one project. Any seed that resolves to a different project is reported as a `cross_project_seed` issue rather than silently mixed in — this closes the same cross-project identity-confusion class #37 exists to prevent. - **Every seed miss is reported, never silent** — `seed_issues` carries missing / ambiguous / cross-project / no-match-for-task-term, each with its offending input. - **Seeds:** `symbol_ids` (generation-local), `handles` (durable, resolved via #37), `paths` (file → symbols, capped `PATH_SYMBOL_CAP=64`), `diff_base` (changed symbols), and **`task_terms`** — lexical focus terms each run through `search_symbols`; matches become seeds, no-match → a seed issue (NL extraction stays client-side). - **Budget classes:** targets are always kept; dependents are ranked by path-relevance and trimmed to fit; witness paths and test-role files included. One dense file can't consume the whole budget. - **Honest accounting:** `tokens_used` is the *true serialized size* of the response (`serde_json::to_string(&resp).len()/4`), and a `budget_exceeded` flag is set rather than silently truncating past the ceiling. `MAX_SEEDS=128`. Every returned item carries an inclusion reason, score, and (for symbols) a `stable_handle`. Dependents are deduped across chunks. ## Benchmark Against the current multi-tool workflow (self-index): context assembly is **1 pack call vs a 5-call manual sequence** (search → get_symbol → find_callers → dependents → tests). The pack front-loads all evidence — targets + ~40 dependents + ~21 test-role files — into one larger payload: a clear round-trip win, with an honest byte trade-off surfaced via `budget_exceeded` and true-size `tokens_used`. ## Review Deep adversarial review (17 agents: 11 confirmed / 2 refuted), all confirmed fixed with regression tests. Headline: cross-project seed-id mixing (project-local numeric ids from different projects) — now single-project-disciplined with `cross_project_seed` reporting. ## Deferred to v2 (this issue stays open) - Analogue expansion (#27) — pull in structurally-similar exemplars beyond direct dependents. - Cycle-aware neighborhood (#28). - Richer personalized-rank witness variety and span-dedup across overlapping ranges.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

h-dv/code-index#35

No description provided.

Rows
Columns