Promotion lock: carry-row redesign shipped; quiet paired benchmark acceptance remains #307

Closed
opened 2026-09-26 22:26:37 +02:00 by buildagent · 3 comments
Member

Audit on v0.41.45 (2026-11-08): the carry-row implementation is already shipped through m0075/LIVE. It must not be redesigned or counted as a new feature. The current 100k-file benchmark and reader/generation controls pass; full 0.41.45 qualification and release are green. The open acceptance item is the original quiet paired measurement, not a currently reproduced 13.9-second carry rewrite. Historical measurements and their contention limits remain preserved below and in the comments.


Found during Phase 1 of the pluggable-languages work (lane R, 2026-09-26). This must be resolved before a language flips to a package, because every package extraction then goes through generation promotion.

State: at 100k files, generation promotion holds the write lock for about 13.94 s (bench_promotion_lock, nightly). The existing record attributes 89.9% of that to "the carry": the UPDATE that re-stamps generation_id on every carried row, which cascades by trigger into symbols, refs and imports.

Direction: do not re-stamp carried rows. For example, keep the active generation's id and move only the dirty rows. This is a schema and read-path redesign across the ~58 places that filter on the active generation.

Gates:

  • bench_promotion_lock before and after, run on a quiet machine.
  • Corpus suites green, with no baseline moved.
  • Every read path that filters on the active generation is covered by a test that fails if it reads a stale generation.
Audit on v0.41.45 (2026-11-08): the carry-row implementation is already shipped through m0075/LIVE. It must not be redesigned or counted as a new feature. The current 100k-file benchmark and reader/generation controls pass; full 0.41.45 qualification and release are green. The open acceptance item is the original quiet paired measurement, not a currently reproduced 13.9-second carry rewrite. Historical measurements and their contention limits remain preserved below and in the comments. --- Found during Phase 1 of the pluggable-languages work (lane R, 2026-09-26). This must be resolved before a language flips to a package, because every package extraction then goes through generation promotion. **State:** at 100k files, generation promotion holds the write lock for about 13.94 s (`bench_promotion_lock`, nightly). The existing record attributes 89.9% of that to "the carry": the UPDATE that re-stamps `generation_id` on every carried row, which cascades by trigger into symbols, refs and imports. **Direction:** do not re-stamp carried rows. For example, keep the active generation's id and move only the dirty rows. This is a schema and read-path redesign across the ~58 places that filter on the active generation. **Gates:** - `bench_promotion_lock` before and after, run on a quiet machine. - Corpus suites green, with no baseline moved. - Every read path that filters on the active generation is covered by a test that fails if it reads a stale generation.
Author
Member

Review found this implementation already delivered by m0075; the current resolver uses the stable LIVE label and preserves carried rows. The existing design/mutation record is _prdoc/records/T-307-promotion-lock.md.

Revalidated the existing release-profile 100,000-file benchmark on 544cf25f (PR #508), without source changes: 100,002 files, 1,000,000 symbols, 1,300,000 refs, 100,000 imports, 2,500,002 generation rows. Empty promotion/rollback held the write lock for 5.2/4.8 ms. With 1,000 dirty files and 23,977 moved rows, promotion/rollback took 319.1/142.7 ms. All benchmark assertions passed. Final load samples were 1.99, 1.91 and 1.84 on this 12-core machine; no competing local test/build ran during measurement. These are current absolute measurements, not a newly claimed before/after speed ratio.

The generation-build/reader registry suite also passes all 40 checks, including the ban on production readers using the active generation id; generation isolation passes 21 checks. Full Linux/native-Windows qualification of the combined release is still running. Local retained receipt: /tmp/cosi-045-promotion-100k/.

Review found this implementation already delivered by m0075; the current resolver uses the stable LIVE label and preserves carried rows. The existing design/mutation record is `_prdoc/records/T-307-promotion-lock.md`. Revalidated the existing release-profile 100,000-file benchmark on 544cf25f (PR #508), without source changes: 100,002 files, 1,000,000 symbols, 1,300,000 refs, 100,000 imports, 2,500,002 generation rows. Empty promotion/rollback held the write lock for 5.2/4.8 ms. With 1,000 dirty files and 23,977 moved rows, promotion/rollback took 319.1/142.7 ms. All benchmark assertions passed. Final load samples were 1.99, 1.91 and 1.84 on this 12-core machine; no competing local test/build ran during measurement. These are current absolute measurements, not a newly claimed before/after speed ratio. The generation-build/reader registry suite also passes all 40 checks, including the ban on production readers using the active generation id; generation isolation passes 21 checks. Full Linux/native-Windows qualification of the combined release is still running. Local retained receipt: `/tmp/cosi-045-promotion-100k/`.
Author
Member

The v0.41.45 qualification is now green on source 3842ab6d: Linux, native Windows, tier-1 corpus, daemon transport and C#/timeline compatibility, followed by all ten release jobs and nine published-artifact acceptance groups. The carry-row redesign was already implemented through m0075/LIVE; this release does not introduce it again.

The fresh current-version 100k-file benchmark and reader/generation controls reported above pass. The historical paired measurements were load-contended, and the fresh reading is not a new quiet before/after pair. Keeping the original quiet paired-measurement criterion explicit rather than claiming it was performed.

The v0.41.45 qualification is now green on source 3842ab6d: Linux, native Windows, tier-1 corpus, daemon transport and C#/timeline compatibility, followed by all ten release jobs and nine published-artifact acceptance groups. The carry-row redesign was already implemented through m0075/LIVE; this release does not introduce it again. The fresh current-version 100k-file benchmark and reader/generation controls reported above pass. The historical paired measurements were load-contended, and the fresh reading is not a new quiet before/after pair. Keeping the original quiet paired-measurement criterion explicit rather than claiming it was performed.
buildagent changed title from Promotion lock at 100k files: stop re-stamping carried rows (13.9 s, 89.9% is the carry) to Promotion lock: carry-row redesign shipped; quiet paired benchmark acceptance remains 2026-11-08 13:54:47 +01:00
Author
Member

Triage 2026-11-09: FIXED: the carry-row redesign is in the tree (contributions::LIVE, _prdoc/records/T-307-promotion-lock.md, 37e59ebd/c4dac2ce/65a43da8). bench_promotion_lock runs nightly: 319 ms at 1,000 dirty files (originally 13.9 s). The 'paired before/after' would compare against a build whose migration chain was deleted (7e8d714c), so it is moot.

Triage 2026-11-09: FIXED: the carry-row redesign is in the tree (`contributions::LIVE`, `_prdoc/records/T-307-promotion-lock.md`, 37e59ebd/c4dac2ce/65a43da8). `bench_promotion_lock` runs nightly: 319 ms at 1,000 dirty files (originally 13.9 s). The 'paired before/after' would compare against a build whose migration chain was deleted (7e8d714c), so it is moot.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
h-dv/code-index#307
No description provided.