Package cold cost is 1.8–2.3x the builtin against a 1.20 budget; python package leaves get_version/create_app/request unresolved (nightly package-ci red) #347

Closed
opened 2026-10-09 07:03:39 +02:00 by buildagent · 2 comments
Member

Found by the CI-FAST lane (#344) while investigating why package-ci is red. These are product defects, not CI flakiness.

1. The cold-cost budget fails for every package

its_package_matches_native_semantics_and_fits_cold_and_warm_budgets (crates/first-party-tests/tests/support/language_overhead.rs:219) already compares the package against the builtin in alternating rounds within one process and takes the median. The comparison is therefore not distorted by contention on the machine.

Measured on a nearly idle box (load 2.6–3.7, 12 cores), every language's package is about 1.8–2.3x the builtin cold, against a budget of 1.20:

  • ruby: 2.31 (nightly CI showed 2.27);
  • python: 3.6.

The scheduled master run 1018 (04b5b96e) failed this test for csharp, javascript, php, python, rust, typescript and ruby. Since v0.33.0 the cold cost was measured at 1.37–3.64x (see the v0.33 notes). The budget has been red since then, and nightly runs carried that as noise.

Decision needed: reduce the package cold cost, or re-derive the budget from measurement with a stated reason. Do not keep a gate that is always red.

2. Python package semantic parity defect (pre-existing)

The same test's parity half fails for python. The package leaves get_version, create_app and request unresolved where the builtin resolves them. It reproduces identically in nightly 1018, in PR run 1041, and locally.

Related: #344, #315 (python package parity), #345.

Found by the CI-FAST lane (#344) while investigating why package-ci is red. **These are product defects, not CI flakiness.** ## 1. The cold-cost budget fails for every package `its_package_matches_native_semantics_and_fits_cold_and_warm_budgets` (`crates/first-party-tests/tests/support/language_overhead.rs:219`) already compares the package against the builtin in alternating rounds within one process and takes the median. The comparison is therefore not distorted by contention on the machine. Measured on a nearly idle box (load 2.6–3.7, 12 cores), every language's package is about **1.8–2.3x the builtin cold**, against a budget of **1.20**: - ruby: 2.31 (nightly CI showed 2.27); - python: 3.6. The scheduled master run 1018 (04b5b96e) failed this test for csharp, javascript, php, python, rust, typescript and ruby. Since v0.33.0 the cold cost was measured at 1.37–3.64x (see the v0.33 notes). The budget has been red since then, and nightly runs carried that as noise. **Decision needed:** reduce the package cold cost, or re-derive the budget from measurement with a stated reason. Do not keep a gate that is always red. ## 2. Python package semantic parity defect (pre-existing) The same test's parity half fails for python. The package leaves `get_version`, `create_app` and `request` unresolved where the builtin resolves them. It reproduces identically in nightly 1018, in PR run 1041, and locally. Related: #344, #315 (python package parity), #345.
Author
Member

Item 2 (python parity): root-caused and fixed in PR #344

Root cause. The defect is in the host, in crates/indexer/src/python_imports.rs build.

  • The python package's layout hook answers ready for every file. On python-flask, pyproject.toml is metadata-only, so the identities it reports are exactly the native path-derived names.
  • The host still marked every file as layout-owned. That switched on the denied-refs arm, which blocks the native tiers' cross-file binds.
  • Result: 91 binds lost, 0 gained. The names were request, app_ctx, g, get_db, json, session, create_app, get_version and others.
  • Correction to #315: the member_access → type kind change follows from the lost bind; it is not a separate extractor defect.
  • Not the cause: #339's profile decoding is not involved.

Fix (910353e9). If a layout restates exactly the native identities for every file, the pass resolves as it would with no layout. The rule is generic, keyed on the import-system profile rather than on python, and decided over the whole layout, never file by file.

  • New test: python_imports::module_metadata_tests::a_layout_restating_the_native_identities_resolves_as_no_layout.
  • The lane ran three mutations against it, all red. I re-ran the restates_native mutation and it was red.
  • Parity is exact. Corpus records did not move; the 11 CI corpus suites and precision_gate (7/7) are green.

Finding, not fixed here. Many of the builtin's own binds that parity reproduces are wrong:

  • get_version binds src/flask/cli.py although it comes from pallets_sphinx_themes;
  • app_ctx binds the pytest fixture;
  • request, g and session bind ctx.py properties instead of the proxies in globals.py.

#336 removes part of this class.

Item 1 (cold-cost budget): still open

  • Python's cold ratio measured 3.47–3.91 against a budget of 1.20, at load 4–6.
  • PR runs only report the ratio; the nightly asserts it, so the nightly stays red on this item.
  • The debug cranelift optimisation in #344 fixed the precompile timeouts. It did not bring the ratio into budget.
## Item 2 (python parity): root-caused and fixed in PR #344 **Root cause.** The defect is in the host, in `crates/indexer/src/python_imports.rs` `build`. - The python package's layout hook answers `ready` for every file. On python-flask, `pyproject.toml` is metadata-only, so the identities it reports are exactly the native path-derived names. - The host still marked every file as layout-owned. That switched on the denied-refs arm, which blocks the native tiers' cross-file binds. - Result: **91 binds lost, 0 gained.** The names were `request`, `app_ctx`, `g`, `get_db`, `json`, `session`, `create_app`, `get_version` and others. - **Correction to #315:** the `member_access` → `type` kind change follows from the lost bind; it is not a separate extractor defect. - **Not the cause:** #339's profile decoding is not involved. **Fix (`910353e9`).** If a layout restates exactly the native identities for every file, the pass resolves as it would with no layout. The rule is generic, keyed on the import-system profile rather than on python, and decided over the whole layout, never file by file. - New test: `python_imports::module_metadata_tests::a_layout_restating_the_native_identities_resolves_as_no_layout`. - The lane ran three mutations against it, all red. I re-ran the `restates_native` mutation and it was red. - Parity is exact. Corpus records did not move; the 11 CI corpus suites and `precision_gate` (7/7) are green. **Finding, not fixed here.** Many of the builtin's own binds that parity reproduces are wrong: - `get_version` binds `src/flask/cli.py` although it comes from `pallets_sphinx_themes`; - `app_ctx` binds the pytest fixture; - `request`, `g` and `session` bind `ctx.py` properties instead of the proxies in `globals.py`. #336 removes part of this class. ## Item 1 (cold-cost budget): still open - Python's cold ratio measured 3.47–3.91 against a budget of 1.20, at load 4–6. - PR runs only report the ratio; the nightly asserts it, so the nightly stays red on this item. - The debug cranelift optimisation in #344 fixed the precompile timeouts. It did not bring the ratio into budget.
Author
Member

Released in v0.36.0.

  • Package discovery reads each package once, in parallel: about 1.1 s → 0.2–0.4 s.
  • A fresh index extracts each file once.
  • The overhead budget is now 1.20 × builtin + 0.5 s, and every first-party package meets it.

🤖 Generated with Claude Code

Released in v0.36.0. - Package discovery reads each package once, in parallel: about 1.1 s → 0.2–0.4 s. - A fresh index extracts each file once. - The overhead budget is now 1.20 × builtin + 0.5 s, and every first-party package meets it. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
h-dv/code-index#347
No description provided.