feat: shrinker — bisect a corpus finding into a minimal fixture + generated oracle.toml probe #48

New issue

Open

opened 2026-07-28 20:39:43 +02:00 by buildagent · 0 comments

buildagent commented

2026-07-28 20:39:43 +02:00

Member

P3 — build when the first real corpus finding lands; premature before then.

From _prdoc/records/brainstorm-2026-07-28-oss-corpus-test-system.md §5.2.

The problem it solves

A phantom in a 5000-file repo is nearly impossible to reason about, and without this the finding stays only in the corpus — a slow test forever, re-run on every CI pass, debugged from scratch every time.

What

Given (repo, ref site, wrong target): bisect the file set — delete half, re-index, does it still reproduce — down to the minimal reproducing set. Emit a new micro-fixture under tests/fixtures/ plus a generated oracle.toml probe in the existing schema (name/kind/path/container/find/expect_resolved/forbid_resolved).

The division of labour it establishes

The corpus is the discovery engine. The fixtures are the regression suite.
Nothing found in the corpus is allowed to stay only in the corpus.

This is what keeps PR CI fast permanently: the lasting cost of each finding becomes a handful of files, not a repository. It also preserves the existing precision_gate.rs as the fast gate rather than growing a parallel slow one.

Design notes

bisection must preserve indexability — deleting a file that the repro depends on ends that branch; standard delta-debugging applies
the generated probe must satisfy the existing I033 anti-vacuity guard: at least one forbid_resolved probe whose (name, kind) resolves to exactly one in-project definition, or the generated oracle is vacuous by construction
emit the note field explaining provenance (source repo, sha, original site) so the fixture is traceable back to the real code

Acceptance

shrinker reduces a seeded known finding to a minimal file set
emits a fixture + valid oracle.toml probe that the existing precision_gate.rs picks up unmodified
generated oracle passes the I033 single-candidate anti-vacuity requirement
provenance recorded in the fixture
round-trip proven on a deliberately injected defect

**P3 — build when the first real corpus finding lands; premature before then.** From `_prdoc/records/brainstorm-2026-07-28-oss-corpus-test-system.md` §5.2. ## The problem it solves A phantom in a 5000-file repo is nearly impossible to reason about, and without this the finding stays *only* in the corpus — a slow test forever, re-run on every CI pass, debugged from scratch every time. ## What Given `(repo, ref site, wrong target)`: bisect the file set — delete half, re-index, does it still reproduce — down to the minimal reproducing set. Emit a new micro-fixture under `tests/fixtures/` plus a generated `oracle.toml` probe in the existing schema (`name`/`kind`/`path`/`container`/`find`/`expect_resolved`/`forbid_resolved`). ## The division of labour it establishes > **The corpus is the discovery engine. The fixtures are the regression suite.** > Nothing found in the corpus is allowed to stay only in the corpus. This is what keeps PR CI fast permanently: the lasting cost of each finding becomes a handful of files, not a repository. It also preserves the existing `precision_gate.rs` as the fast gate rather than growing a parallel slow one. ## Design notes - bisection must preserve indexability — deleting a file that the repro depends on ends that branch; standard delta-debugging applies - the generated probe must satisfy the **existing I033 anti-vacuity guard**: at least one `forbid_resolved` probe whose `(name, kind)` resolves to exactly one in-project definition, or the generated oracle is vacuous by construction - emit the `note` field explaining provenance (source repo, sha, original site) so the fixture is traceable back to the real code ## Acceptance - [ ] shrinker reduces a seeded known finding to a minimal file set - [ ] emits a fixture + valid `oracle.toml` probe that the existing `precision_gate.rs` picks up unmodified - [ ] generated oracle passes the I033 single-candidate anti-vacuity requirement - [ ] provenance recorded in the fixture - [ ] round-trip proven on a deliberately injected defect

buildagent added the

Kind/Feature

Priority

Low

labels

2026-07-28 20:40:54 +02:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

h-dv/code-index#48

No description provided.

Rows
Columns