feat: semantic_diff / review_diff — explain structural risk, not changed lines #34

Open
opened 2026-07-20 22:45:36 +02:00 by buildagent · 2 comments
Member

Product

Turn changed_symbols from a symbol inventory into an agent-ready semantic review of a working tree, staged change, branch, or commit range.

Report

  • added/modified/deleted/renamed symbols and public API surface
  • call/dependency edges added or removed
  • symbols that became unreachable, test-only, or newly exposed
  • transitive impact and affected tests via #22
  • cycles or crate-layer violations introduced via #28
  • new clone/drift relationships via #27
  • resolution-quality delta: newly unresolved/resolved refs and confidence change
  • concise risk-ranked findings with reason codes; detailed evidence on demand

Why

Line diffs answer what text changed. Agents need what the change means: “this private edit reaches three public entry points,” “this deletion orphans two helpers,” or “this new import creates the first reverse crate edge.”

The current changed_symbols already performs the hardest diff-to-symbol mapping, including base-side deleted symbols. The missing layer is composition over existing indexed analyses, with stable finding IDs and honest confidence.

Design

This should be an orchestrator, not a second implementation of every analysis:

  1. obtain seeds and base/current symbol identities from changed_symbols
  2. compare graph/API states where data is available
  3. call shared engines from #22, #27, and #28
  4. rank findings by reach, visibility, boundary crossing, test coverage, and confidence
  5. render a bounded summary plus drill-down handles

Renames should be reported as inferred pairs with confidence, while preserving added+deleted truth underneath. Do not claim behavioral equivalence from matching names alone.

Acceptance

  • supports working tree, staged, and arbitrary commit-ish base
  • deleted symbols retain base-side impact evidence where possible
  • deterministic risk ordering and stable machine-readable reason codes
  • every negative claim carries graph coverage/confidence
  • partial analyzers fail independently and are listed as skipped, not hidden
  • concise output fits normal review context; pagination/drill-down handles retain evidence
  • integration fixtures cover API break, orphan creation, new cycle, missing test, and resolution regression
## Product Turn `changed_symbols` from a symbol inventory into an agent-ready semantic review of a working tree, staged change, branch, or commit range. ### Report - added/modified/deleted/renamed symbols and public API surface - call/dependency edges added or removed - symbols that became unreachable, test-only, or newly exposed - transitive impact and affected tests via #22 - cycles or crate-layer violations introduced via #28 - new clone/drift relationships via #27 - resolution-quality delta: newly unresolved/resolved refs and confidence change - concise risk-ranked findings with reason codes; detailed evidence on demand ## Why Line diffs answer **what text changed**. Agents need **what the change means**: “this private edit reaches three public entry points,” “this deletion orphans two helpers,” or “this new import creates the first reverse crate edge.” The current `changed_symbols` already performs the hardest diff-to-symbol mapping, including base-side deleted symbols. The missing layer is composition over existing indexed analyses, with stable finding IDs and honest confidence. ## Design This should be an orchestrator, not a second implementation of every analysis: 1. obtain seeds and base/current symbol identities from `changed_symbols` 2. compare graph/API states where data is available 3. call shared engines from #22, #27, and #28 4. rank findings by reach, visibility, boundary crossing, test coverage, and confidence 5. render a bounded summary plus drill-down handles Renames should be reported as inferred pairs with confidence, while preserving added+deleted truth underneath. Do not claim behavioral equivalence from matching names alone. ## Acceptance - supports working tree, staged, and arbitrary commit-ish base - deleted symbols retain base-side impact evidence where possible - deterministic risk ordering and stable machine-readable reason codes - every negative claim carries graph coverage/confidence - partial analyzers fail independently and are listed as skipped, not hidden - concise output fits normal review context; pagination/drill-down handles retain evidence - integration fixtures cover API break, orphan creation, new cycle, missing test, and resolution regression
Author
Member

Minimal v1 shipped in v0.6.0 (I030 Track C) as review_diff — the narrow scope this issue's roadmap called for. changed_symbols' classification core extracted (collect_changed, zero behavior change, 19 existing tests green) and composed with one change_impact call: risk-ranked findings with stable reason codes (api_break_deleted with live-breakage evidence, untested_change as a disclosed static proxy, wide_blast with witness chain, low_confidence_area self-disclosure, seed_overflow), severity-sorted, capped at 50, confidence + graph_semantics riding along. Deferred to a future round per the spec's larger vision: rename-pair inference, cycle/clone analyzers (#27/#28 not yet shipped), per-seed blast granularity. Bench: review + test-selection = 2 calls / 4.3KB on this repo (21 findings, 21-file test selection).

Minimal v1 shipped in **v0.6.0** (I030 Track C) as `review_diff` — the narrow scope this issue's roadmap called for. `changed_symbols`' classification core extracted (`collect_changed`, zero behavior change, 19 existing tests green) and composed with one `change_impact` call: risk-ranked findings with stable reason codes (`api_break_deleted` with live-breakage evidence, `untested_change` as a disclosed static proxy, `wide_blast` with witness chain, `low_confidence_area` self-disclosure, `seed_overflow`), severity-sorted, capped at 50, confidence + `graph_semantics` riding along. Deferred to a future round per the spec's larger vision: rename-pair inference, cycle/clone analyzers (#27/#28 not yet shipped), per-seed blast granularity. Bench: review + test-selection = 2 calls / 4.3KB on this repo (21 findings, 21-file test selection).
Author
Member

Triage: P1 — keep open as review_diff v2

v1 shipped in v0.6.0, and the production-hardening follow-up changed two details from the existing shipment comment:

  • all changed seeds are now analyzed in deterministic batches; seed_overflow was removed
  • private deletions no longer produce API-break findings, and test-role seeds no longer produce untested_change

Recommended v2 scope:

  • stable finding/evidence handles via #37
  • per-seed impact and base-side evidence for deletions
  • optional task-intent correlation and executed-test input
  • graph/API edge deltas
  • optional #27/#28 analyzers that fail independently
  • validation suggestions with evidence, not semantic-correctness claims

Prefer evolving this tool over adding a separate validate_change tool.

### Triage: P1 — keep open as `review_diff` v2 v1 shipped in v0.6.0, and the production-hardening follow-up changed two details from the existing shipment comment: - all changed seeds are now analyzed in deterministic batches; `seed_overflow` was removed - private deletions no longer produce API-break findings, and test-role seeds no longer produce `untested_change` Recommended v2 scope: - stable finding/evidence handles via #37 - per-seed impact and base-side evidence for deletions - optional task-intent correlation and executed-test input - graph/API edge deltas - optional #27/#28 analyzers that fail independently - validation suggestions with evidence, not semantic-correctness claims Prefer evolving this tool over adding a separate `validate_change` tool.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
h-dv/code-index#34
No description provided.