CI fast path: nextest, bump-only skip, package releases without a host round, fast_gates.sh (#318) #344
No reviewers
Labels
No labels
code-review
correctness
dos
performance
security
severity/high
severity/low
severity/medium
tech-debt
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
h-dv/code-index!344
Loading…
Reference in a new issue
No description provided.
Delete branch "ci-fast"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This PR answers the operator's request: "we urgently need to reduce CI time and be more productive", and "every change need test, PR, merge, release. this for 4x2hours for every simple EBF." It also serves the goal of language packages: "a faster development process and better maintenance, since we can deploy independent".
Measured before (round on the 0.34.0 commit)
cargo testrunning 428 binaries one at a time; ~15 min of rebuilds after the testscargo testCommits
bb7e59d2: a package release needs only its own CI legs.package-release.ymlrequired the host jobs to be green. A package-only commit has no host rows, so every package release would have waited 50 min and then needed a hand-dispatched ~85 min host round. Nopkg/*tag had ever been pushed, so this never surfaced.package_ci_gate.shrequires the package's own two legs to be green and the host jobs only to be "not red". This reverses a #309 rule; the operator approved it on 2026-10-08.package (ruby)(dev) andpackage corpus (ruby)(release), plus one sharedlintjob. No gate was removed.tests/packages/README.md.3da1ec61andd1d86fe6: a version-bump-only commit costs no CI round.BUMP_SITESdeclared inversion_site_registry.rs, read from the parent.*.rsand CI files can never be skippable sites.require_ci_greendecides on the tagged commit's own rows first.d1d86fe6closes a "ci-gate-bypass" finding from the automated security review. I mutated the.rsrefusal myself, andversion_only_means_the_version_string_and_nothing_elsewent RED.99c4edc4: the workspace suite runs undercargo nextest(profileci).cargo test --doc; the-- --ignoredsuites stay oncargo test.language_package_overhead, whose 30 s precompile budget is starved on a loaded box. It andhost_extraction_epochare serialised in a test group.942a3861:fast_gates.shruns all 61 tree-reading and registry gates in about 2 min.fast_gates_registry.rskeeps the list complete. Lanes run it before reporting; today a lane missed three such gates.0dd5cf89: the Windows job sets oneRUSTFLAGS, so the second full dev build after the tests is gone.This PR's own CI is the first real-runner test of
RUSTFLAGSchange;Not in this PR (needs the operator)
target/on the runner would fix it, but it has the stale-build-hash hazard.🤖 Generated with Claude Code
https://claude.ai/code/session_0126PDDLB4wNHxKXvWM1VNmu
ci.yml and ci-windows.yml run `cargo nextest run --workspace --exclude code-index-first-party-tests --no-fail-fast --profile ci` and, beside it, `cargo test --doc` over the same set (nextest runs no doc-tests). The `-- --ignored` suites stay on `cargo test`: they are timing and corpus measurements that run one at a time on purpose. MEASURED locally, same args, same warm build, 12 cores (shared box): cargo test 2857 s (47.6 min); load1 at start 4.18, median 4.2 cargo nextest 782 s (13.0 min); load1 at start 26.1 (15-min 8.3), median 15.9 (nextest's own parallelism) 3.7x. One test failed under nextest only: language_package_overhead::php_psr4_layout_adds_no_reachability_the_ builtin_lacks — its package refused with PRECOMPILE_WALL_CLOCK_BUDGET_MS observed 30000: a debug cranelift precompile starved of CPU by the other binaries (the whole binary took 33 s under cargo test). Those precompile-heavy binaries are a nextest test-group (max 2, 4 slots each). The group config was run on its two binaries (13 passed, 176 s); a full re-run under it was NOT made (two full runs was the budget). .config/nextest.toml `ci` profile: retries = 0 (a flake stays red), failure output immediate+final, slow tests reported every 120 s and never killed. Install: the Linux image ci-rust:1.94.0-v7 ships cargo-nextest 0.9.144 (probed); the step prints its version so an image without it fails by name. Windows: a pinned 0.9.144 prebuilt zip, sha256 measured here (upstream publishes none), skipped when that version is present. Gates still grade the real invocation: ci_path_gating's package-home rule reads `cargo nextest run` as a test invocation (and now expects four host workspace runs), ci_cadence counts a nextest step as a grading step and requires --profile ci and retries = 0, ci_disk_preflight treats `cargo nextest` as a compiler invocation. Mutations run: retries = 2, --profile ci dropped, --exclude dropped, the doc-test step's `success() || failure()` dropped — each RED. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0126PDDLB4wNHxKXvWM1VNmuWHY THE JOB REBUILT (run 1022, measured from its log). After 61 min of tests the job compiled the workspace four more times: 11:40 46 s test profile `cargo test -p code-index-cli --test installer_ps1_e2e`: `-p` alone changes feature unification against the `--workspace` build (NOT changed here; ~46 s) 11:43 3m08s dev `cargo build --workspace --bins` under 11:45 1m47s dev RUSTFLAGS=+crt-static (+ release_smoke): RUSTFLAGS is in every crate's fingerprint, so a full second dev graph 11:51 5m11s release the release worker + smoke harness (#232), 11:55 3m51s release under the same flags The job-level RUSTFLAGS removes the second dev graph (~5 min): clippy, build and test now build what the smoke steps build, and the tests run the CRT linkage the archive ships. The release-profile smoke is KEPT: it is the per-push evidence about the profile the archive ships, and moving it to release.yml would trade ~9 min here for finding #232's class of defect at tag time. Also found: the job's actions/cache never restores ("Cache not found" for every key) — its post-step tars a 42 GB target/ — so every run builds cold. Not changed here: a persistent CARGO_TARGET_DIR on the runner is the obvious fix and is exactly the stale-build-hash hazard recorded for shared target dirs; it needs the operator. UNVERIFIED ON WINDOWS: not dispatched (per instructions). release.yml's build-windows already builds the full workspace, proc-macros included, under this exact flag, which is the evidence it compiles; whether every test passes under the static CRT is what the first dispatch will show. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0126PDDLB4wNHxKXvWM1VNmuEvery test that runs a real package precompiles its extractor through the DEBUG `code-index-plugin-host --precompile`, and unoptimised cranelift was the cost: MEASURED 8.0-10.2 s of CPU per first-party extractor, single-threaded (load 8.4, 12 cores). The product kills a precompile at PRECOMPILE_WALL_CLOCK_BUDGET_MS = 30 s, so a busy CI box ran debug tests into it and the package was refused as `extractor.malformed`: - package-ci run 1041, `package (python)`: package_release_scripts' `plugin check` verdict "staging REFUSED extractor.malformed"; - the local nextest run of language_package_overhead (observed: 30000). Not caused by the CI-FAST E logic, but made likely by it: the dev/corpus split doubled the concurrent jobs on the package box (21 in run 1041). opt-level = 3 for the cranelift crates, regalloc2 and wasmtime-internal-cranelift only; every other crate's debug build is unchanged and the product bound is unchanged. MEASURED after: precompile, six extractors 0.7-0.9 s each (was 8.0-15.4 s) host_extraction_epoch + language_package_overhead under nextest 54.8 s, load 5.1 (was 176 s, load 6.3) package_release_scripts 92.8 s, load 2.6 (cargo test: 337 s, load ~4; nextest full run: 437 s, contended) one-off cost: 53 s incremental rebuild of the plugin host. ci_cadence::the_wasm_code_generator_is_optimised_in_debug_builds reads the crate set from Cargo.lock (a wasmtime upgrade that adds one fails until it is optimised); it found cranelift-native, which the first draft missed. Mutation RUN: drop the cranelift-codegen override — RED naming it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0126PDDLB4wNHxKXvWM1VNmuDECISION (b), from run 1041 against nightly 1018 — both "every package in scope": 1018, one job per package: legs 43.8-50.8 min, wall ~51 min 1041, dev + corpus split: dev 35.8-42.3, corpus 52.5-58.5, lint 18.5; wall 58.5 min, 21 heavy jobs at once, python's precompile ran into the product's 30 s bound Every change to package-ci.yml, crates/first-party-tests or a shared guest crate puts every package in scope, as does the nightly, so on the one package box the split was a net LOSS of ~8 min and a source of contention failures. The leg is one job again; the once-per-run `lint` job stays (it lints the shared crate, not a package). What the split was meant to buy comes from the SELECTION. Each cargo package selection is its own feature unification, so its own build of the dependency graph; the leg used four dev selections and two release ones (1041 python: 17m37 + 7m03 + 7m14 dev, 27m04 + 22m49 release of compiling). Every dev `cargo test` now names the same four packages, and every release `cargo test`/`cargo build` the same two. MEASURED locally from an empty target dir, the dev steps only (load 7.6-11): old selections 81 + 65 + 39 + 28 s = 213 s, 406 crates compiled one selection 67 + 0 + 11 + 1 s = 79 s, 267 crates (The in-test `cargo build -p code-index-plugin-host` of package_release_scripts still builds its own variant: 15-43 s.) package_ci_gate.sh requires the one leg `package (<name>)`. ci_path_gating::a_leg_builds_each_profile_once replaces the two-halves test. Mutations RUN, each RED: drop `-p code-index-plugin-host` from the phase 3/4 step (two dev selections); restore the plugin-host-only release build (two release selections); require the old corpus leg name in the gate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0126PDDLB4wNHxKXvWM1VNmuRun 1045: m2_acceptance::m2_thousand_file_repo_finishes_quickly asserted a 1000-file cold index under 6 s (its own comment: ~120 ms in a debug build) and took 21.45 s beside the nextest suite on a box running ten package jobs. The bound was right; the measurement was contended. THE POPULATION, by rule rather than by name: crates/test-support/tests/wall_clock_registry.rs lexes every #[test] / #[tokio::test] in crates/*/{src,tests,benches,examples} and takes the ones that assert an UPPER bound on a value derived from `.elapsed()` (directly or through a `let` chain: `took`, `ratio`, ...), including `if took > BUDGET { panic!() }`. Lower bounds ("it really waited") and deadline loops (`assert!(Instant::now() < deadline)`) are left out. 31 non-ignored tests in 6 packages, m2 among them; #[ignore]d ones run under `cargo test -- --ignored --test-threads=1` already. It cannot see a bound asserted inside a helper the test calls. THE MECHANISM: `[profile.ci]` gets `default-filter = 'not (…)'` over those 31, and `[profile.ci-timing]` runs exactly them with `test-threads = 1`, as a second nextest step in ci.yml and ci-windows.yml (`success() || failure()`). No bound was raised. NOT an exclusive override inside one run, and that was MEASURED: nine binaries / 284 tests under 12 busy loops of external load, threads-required = "num-test-threads" override 676 s no override 244 s ci (excluding them) + ci-timing 230 + 84 = 314 s because nextest drains every running test before each exclusive one. m2 under that load: 3.6-4.0 s beside the subset, 2.0 s alone. External load (other jobs on the CI box) is not removed by any nextest setting. Gates and mutations RUN, each RED: wall_clock_registry: drop m2 from both profiles (named missing); test-threads = 4; a stale term in both; m2 dropped from ci's exclusion only (the profiles disagree); synthetic take/leave controls. ci_cadence: delete ci-windows.yml's ci-timing step (the excluded tests would run nowhere). fast_gates.sh lists the new registry (fast_gates_registry requires it): 62 targets, 488 tests, 79 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0126PDDLB4wNHxKXvWM1VNmuFast-forwarded into master at
05dcbee2, approved by the operator. Round 5 (runs 1050–1052) was green on every job. This is a CI-only change, so there is no release.Measured on the real runners:
cargo test: 29.6 min on a quiet box, 56 min with the box overloaded.Also in this PR:
910353e9.#347 item 1, the python cold-cost budget, stays open. The nightly run asserts it.
Forgejo does not mark a fast-forward as merged, so I'm closing this by hand.
Pull request closed