Generation rollback outcome depends on SQLite rowid reuse: an edited file can lose its live rows #321
Labels
No labels
code-review
correctness
dos
performance
security
severity/high
severity/low
severity/medium
tech-debt
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
h-dv/code-index#321
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Found while root-causing a CI-only failure of
generation_promotion::a_rollback_skips_queued_files_that_no_longer_exist. The test is fixed; this product defect is not.Mechanism
files.idis a plainINTEGER PRIMARY KEY, not AUTOINCREMENT (crates/indexer/src/migrations/m0001_initial.rs:5).write_upsertreplaces a file's row with a DELETE followed by an INSERT (crates/indexer/src/writer.rs:306-340).max(rowid)+1. If the edited file held the highest id, it gets the SAME id back.walker.rs:967,index.rs:574-611).Effect, measured with debug output in the test
src/notes.mdends with no live contribution at all.So the rollback's result depends on rowid allocation. The cause is that
generation_build_files.file_idis assumed never to be reissued (generations.rs:1168-1172, withdraw at:1003-1016). A reissued id can also name a DIFFERENT path: delete the highest-id file, add a new file, and the new file takes the old id.Separately,
write_upserthas no "retained rows" branch likewrite_delete's, so editing a file also deletes the old generation's retained rows.Reproduce: check out the test before its fix (PR #320), run it with
TMPDIR=/dev/shm/...andRAYON_NUM_THREADS=1. It fails 10 out of 10 times.Fix options (design decision needed):
files.idAUTOINCREMENT. That is a table rebuild, with the known parent-table cascade hazard.file_id.Either way, add a test that stages the reuse deliberately rather than by luck.
Current-source audit on
b7c34798(isolated worktree; no release source changes): both forced reuse cases pass. With the edited path as the only indexed file, SQLite demonstrably reused id 1 → 1; package activation → edit → rollback → builtin pass matched a cold builtin index of current bytes. A second case deleted the queued path, reconciled to zero files, then created a different path which demonstrably reused id 1; rollback plus the builtin pass again matched the cold whole-tree projection.The current promotion code refreshes generation_build_files.file_id through files.path before withdrawing rows; it also marks missing incoming facts as owed. The original rowid-dependent outcome did not reproduce in either deterministic case. This does not assert retained obsolete rows survive an edit: the checked contract is correct current-byte facts after rollback and convergence.
Keeping this issue open pending a permanent, deliberately forced-reuse regression (the existing owner_switch_rollback edit test exercises the corrected path lookup but does not force SQLite allocation). No AUTOINCREMENT/table rebuild appears necessary for these witnessed cases.
buildagent referenced this issue2026-10-24 22:48:55 +02:00
PR#462 adds the missing permanent forced-reuse regression. Both edited-path and delete/replacement-path cases start from an empty files table, assert actual id reuse, and compare the post-rollback/pass whole projection to a cold index of current bytes. It additionally asserts durable owed work for invalidated retained facts and that rollback does not withdraw the replacement path's current facts. The current path-based queue refresh already handles these cases: disabling that refresh makes this test RED at the replacement's live-contribution assertion. No AUTOINCREMENT/table rebuild is needed for these witnesses. Normal CI is running before closure.