Nightly: language packages exceed the cold/warm overhead budget against their builtins (1.4–2.1x cold) #362

Closed
opened 2026-10-13 06:06:13 +02:00 by buildagent · 0 comments
Member

The nightly package-corpus (<lang>) legs are the only runs that assert the overhead budget: COSI_OVERHEAD_SEMANTIC_ONLY=0 on schedule only (package-ci.yml), checked in crates/first-party-tests/tests/support/language_overhead.rs. The budget is cold ≤ 1.20 × builtin + 0.50 s and warm ≤ 1.10 ×. PR and push runs report the ratios but don't assert them (ratios_asserted: false), so this stays invisible until the nightly.

Semantics are fine: semantic_failure: null on every repo. These are budget failures only.

Measured

Nightly 1117 (master 85b29151, after #354 lane C made every language package own its walker):

repo cold package cold limit cold ratio warm ratio
cs-dapper 5.066 s 4.037 s 1.719 1.322
js-express 3.546 s 2.720 s 1.917 1.107
php-guzzle 6.097 s 5.048 s 1.609 —
python-flask 3.915 s 2.735 s 2.103 1.108
rust-ripgrep 3.901 s 3.872 s 1.388 —
ruby-sinatra — — — 1.136

typescript failed too. Its numbers are in the job log.

Nightly 1103 (876b78b5, the previous release) already failed the same way: cs-dapper cold 1.653 / warm 1.103, and js-express cold 1.714. It isn't new. Lane C only put more languages under the budget.

Caveat: contention

Run 1117 overlapped with the push runs of the merge: package-ci 1116 and ci.yml 1115, the latter's cargo test taking 7302 s. Some cold samples are clearly contended: cs-dapper's first cold package sample was 13.1 s. The medians are still over budget, but the first job is an isolated re-measure (one leg, no parallel runs) before anyone attributes the overhead.

Why it matters

The direction is "all parsers become plugins". A user who switches a language to its package currently pays roughly 1.4–2.1× cold-index time against the builtin.

Next

  1. Re-measure isolated, per language.
  2. Attribute the cold overhead: worker spawn, grammar compile/cache, fact transfer and the resolve delta. diagnostic_cold_breakdown exists in the row for this.
  3. Fix or re-budget with evidence. Never raise the limit without attribution.

Related: the nightly ci.yml worker flake (filed alongside), #354.

The nightly `package-corpus (<lang>)` legs are the only runs that assert the overhead budget: `COSI_OVERHEAD_SEMANTIC_ONLY=0` on `schedule` only (`package-ci.yml`), checked in `crates/first-party-tests/tests/support/language_overhead.rs`. The budget is cold ≤ 1.20 × builtin + 0.50 s and warm ≤ 1.10 ×. PR and push runs report the ratios but don't assert them (`ratios_asserted: false`), so this stays invisible until the nightly. Semantics are fine: `semantic_failure: null` on every repo. These are budget failures only. ## Measured **Nightly 1117** (master 85b29151, after #354 lane C made every language package own its walker): | repo | cold package | cold limit | cold ratio | warm ratio | |---|---|---|---|---| | cs-dapper | 5.066 s | 4.037 s | 1.719 | 1.322 | | js-express | 3.546 s | 2.720 s | 1.917 | 1.107 | | php-guzzle | 6.097 s | 5.048 s | 1.609 | — | | python-flask | 3.915 s | 2.735 s | 2.103 | 1.108 | | rust-ripgrep | 3.901 s | 3.872 s | 1.388 | — | | ruby-sinatra | — | — | — | 1.136 | typescript failed too. Its numbers are in the job log. **Nightly 1103** (876b78b5, the previous release) already failed the same way: cs-dapper cold 1.653 / warm 1.103, and js-express cold 1.714. It isn't new. Lane C only put more languages under the budget. ## Caveat: contention Run 1117 overlapped with the push runs of the merge: package-ci 1116 and ci.yml 1115, the latter's `cargo test` taking 7302 s. Some cold samples are clearly contended: cs-dapper's first cold package sample was 13.1 s. The medians are still over budget, but the first job is an **isolated** re-measure (one leg, no parallel runs) before anyone attributes the overhead. ## Why it matters The direction is "all parsers become plugins". A user who switches a language to its package currently pays roughly 1.4–2.1× cold-index time against the builtin. ## Next 1. Re-measure isolated, per language. 2. Attribute the cold overhead: worker spawn, grammar compile/cache, fact transfer and the resolve delta. `diagnostic_cold_breakdown` exists in the row for this. 3. Fix or re-budget with evidence. Never raise the limit without attribution. Related: the nightly ci.yml worker flake (filed alongside), #354.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
h-dv/code-index#362
No description provided.