feat: provenance-aware gap miner for builtin and plugin evidence #47

New issue

Open

opened 2026-07-28 20:39:29 +02:00 by buildagent · 0 comments

buildagent commented

2026-07-28 20:39:29 +02:00

Member

P3 — the mechanism that turns the corpus into a development engine. Depends on #40, #45.

From _prdoc/records/brainstorm-2026-07-28-oss-corpus-test-system.md §5.1.

What

After each corpus run, aggregate unresolved refs by (lang, reason_code, syntactic shape of the call site) and rank by frequency × repos affected:

lang=ruby   reason=unresolved_bare  shape=block_yield_call    n=1204  repos=3
lang=rust   reason=no_candidate     shape=macro_expanded      n= 890  repos=4
lang=php    reason=unresolved_qual  shape=facade_static_call  n= 655  repos=2  [WONTFIX: unresolvable]

Every row is a mission candidate shipping with its own success metric: fix it, re-run, watch n drop.

Why it matters

This is the I025/I026/I037 process — which has a strong track record — but prioritised by real-world frequency across many codebases instead of by whatever happened to annoy someone while dogfooding this one repo. All three of those missions started from a single anecdotal observation.

The clamp (non-negotiable)

Every recall mission is gated by the Tier-A phantom count staying at 0 (#43, precision_gate.rs). Recall may only rise while phantoms stay at zero. Without that clamp "improve recall" degenerates into "resolve more, guess more" — the exact failure this project has spent I023/I025/I026/I033/I037 avoiding, and the reason the build-vs-buy verdict rejected ctags.

WONTFIX rows are a feature

Laravel facades, Rails method_missing, proc-macro-generated call sites are unresolvable in principle. Marking them as documented ceilings — the same device as oracle.toml's name_fallback_ceiling — keeps them out of the queue permanently and turns the honesty stance into a recorded engineering decision rather than an omission.

Acceptance

miner runs off corpus output, no new schema
ranked table emitted as a CI artifact per run
WONTFIX classification supported with a mandatory reason
each row carries a reproducible measurement so a mission can show its own delta
documented that no recall mission merges with phantom_count > 0

Runtime-plugin architecture expansion

Aggregate by active semantic source, not language string alone:

builtin vs package contribution;
package/component/generation handle;
language profile and granted capability;
bridge/ref class;
dynamic influence grade;
resolver degradation/work-budget state.

A gap caused by a rejected/uninstalled package is coverage work, not resolver recall. A gap caused by a missing capability is a trust/configuration decision. A gap inside a granted, active profile is a resolver/extractor mission.

Never compare or rank counts across different activation generations as if they were one population. Corpus artifacts carry exact package digests and grants. WONTFIX records are scoped to that semantic identity and must be reconsidered when the package/profile changes.

**P3 — the mechanism that turns the corpus into a development engine. Depends on #40, #45.** From `_prdoc/records/brainstorm-2026-07-28-oss-corpus-test-system.md` §5.1. ## What After each corpus run, aggregate unresolved refs by `(lang, reason_code, syntactic shape of the call site)` and rank by frequency × repos affected: ``` lang=ruby reason=unresolved_bare shape=block_yield_call n=1204 repos=3 lang=rust reason=no_candidate shape=macro_expanded n= 890 repos=4 lang=php reason=unresolved_qual shape=facade_static_call n= 655 repos=2 [WONTFIX: unresolvable] ``` Every row is a mission candidate shipping with its own success metric: fix it, re-run, watch `n` drop. ## Why it matters This is the I025/I026/I037 process — which has a strong track record — but prioritised by **real-world frequency across many codebases** instead of by whatever happened to annoy someone while dogfooding this one repo. All three of those missions started from a single anecdotal observation. ## The clamp (non-negotiable) **Every recall mission is gated by the Tier-A phantom count staying at 0** (#43, `precision_gate.rs`). Recall may only rise while phantoms stay at zero. Without that clamp "improve recall" degenerates into "resolve more, guess more" — the exact failure this project has spent I023/I025/I026/I033/I037 avoiding, and the reason the build-vs-buy verdict rejected ctags. ## WONTFIX rows are a feature Laravel facades, Rails `method_missing`, proc-macro-generated call sites are **unresolvable in principle**. Marking them as documented ceilings — the same device as `oracle.toml`'s `name_fallback_ceiling` — keeps them out of the queue permanently and turns the honesty stance into a recorded engineering decision rather than an omission. ## Acceptance - [ ] miner runs off corpus output, no new schema - [ ] ranked table emitted as a CI artifact per run - [ ] WONTFIX classification supported with a mandatory reason - [ ] each row carries a reproducible measurement so a mission can show its own delta - [ ] documented that no recall mission merges with phantom_count > 0 ## Runtime-plugin architecture expansion Aggregate by active semantic source, not language string alone: - builtin vs package contribution; - package/component/generation handle; - language profile and granted capability; - bridge/ref class; - dynamic influence grade; - resolver degradation/work-budget state. A gap caused by a rejected/uninstalled package is coverage work, not resolver recall. A gap caused by a missing capability is a trust/configuration decision. A gap inside a granted, active profile is a resolver/extractor mission. Never compare or rank counts across different activation generations as if they were one population. Corpus artifacts carry exact package digests and grants. WONTFIX records are scoped to that semantic identity and must be reconsidered when the package/profile changes.

buildagent added the

Kind/Feature

Priority

Low

labels

2026-07-28 20:40:53 +02:00

buildagent referenced this issue

2026-07-29 11:53:35 +02:00

gate: generation-aware corpus baselines and reasoned semantic ratchets #45

dhoyer referenced this issue from a commit

2026-08-07 08:21:17 +02:00

test(corpus): treat an AV-deleted tracked file as absent, not silent drift (#47)

buildagent changed title from ~~feat: gap miner — rank unresolved refs into a mission queue clamped by phantom==0~~ to feat: provenance-aware gap miner for builtin and plugin evidence

2026-08-26 13:39:49 +02:00

buildagent referenced this issue

2026-08-26 13:39:49 +02:00

investigate: honest internal-resolution spread across builtin and plugin languages #46

buildagent referenced this issue

2026-08-26 13:40:07 +02:00

gate: generation-aware corpus baselines and reasoned semantic ratchets #45

buildagent added a new dependency

2026-08-26 13:43:48 +02:00

#45 gate: generation-aware corpus baselines and reasoned semantic ratchets

buildagent added a new dependency

2026-08-26 13:43:50 +02:00