Release 0.40.0: repository config is a request (#380), state and DB trusted on proof (#383), live index converges (#381), daemon lifecycle (#382 #358 #387), edits cost what they touch (#384) #388

Merged
buildagent merged 21 commits from train-0400 into master 2026-10-16 09:37:23 +02:00
Member

Safety/integrity train. Five lanes (D, S1, S2, W, P), each verified by the operator's session with its own mutations, then integrated and gated once.

Contents

  • #380: a committed .code-index.toml's [[links]] and a raised max_file_size_mb are requests until granted with the new code-index approve. Built-in parsing has a wall-clock deadline. read_code serves indexed files only. Every git call is built in one place, with repository-configured programs neutralised. PR checkouts no longer persist the token. Both installers are https-only.
  • #383: .code-index/ must be a real, user-owned directory with no links in it. State writes are durable. SQLite runs with TRUSTED_SCHEMA=0. A reset needs every schema object to be a known code-index name. A damaged index in its own state directory is reset in place.
  • #381 / #382.3: each watcher write is validated against disk, and index_freshness self-heals. Fixes a pathless-event panic that killed the watcher thread. The debouncer runs with NoCache (mass deletes are linear).
  • #382 / #358 / #387:
    • A daemon at its cap answers busy and never gets evicted.
    • A spawn poll stops early when the child it spawned has died.
    • daemon.lock is no longer unlinked.
    • plugin rollback and plugin retry exit non-zero under a live daemon.
    • An unchanged restart with the Vue package parses nothing.
    • idle_timeout_min is read from the config.
  • #384: a scoped pass works from a materialised ref set (rust-analyzer body edit 3.5 s → 1.0 s, binds identical on all 9 pinned repos). quick_check runs once per unchanged file. The graph tools walk by frontier. Schema 84: two redundant refs indexes dropped, idx_symbol_edges_to added.

Integration fixes (beyond the lanes)

  • idx_symbol_edges_to added to KNOWN_OBJECTS.
  • The integrity check is skipped only for a file unchanged since it passed (creation time + mtime + length). P's path + ctime key let a database damaged in place after its check open unchecked, contradicting S1's reset test. Measured on a real daemon start: 1 check and 2 skips, so the saving holds.
  • plugin retry gets the same BlockedByOwner treatment as rollback, with a new test (mutation run: red).
  • W's pinned-link fixture grants its link. Lane D's two wall-clock tests run in ci-timing.

Records

  • host-package-lifecycle.json: schema 83 → 84, conditions only (empty categorised diff).

  • ruby-package-cost.json: re-measured under schema 84, operator-approved after per-statement attribution against master. Package vm_step −204,248 (−0.73%):

    • INSERT INTO refs: −213,040 (two fewer index maintenances per row)
    • new reverse edge index: +12,517

    Executions are identical statement by statement. wall_ratio_pct records the bless run's 141 (surveyed 134/139/141).

  • baseline.json and cost-baseline.json are untouched.

Gate

One full local run, including ci.yml's nightly-only jobs: everything green except four steps.

  • Three were integration catches, fixed above. clippy, docs, the workspace leg (5,148), the daemon leg (968) and the timing profile (36) were re-run green.
  • ovh_python: warm ratio 1.103 vs 1.10 at load 2–4.6. It was 1.03–1.04 on three isolated reruns.
  • precision_gate: 7/7, 0 phantoms.

🤖 Generated with Claude Code

https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj

Safety/integrity train. Five lanes (D, S1, S2, W, P), each verified by the operator's session with its own mutations, then integrated and gated once. ## Contents - **#380:** a committed `.code-index.toml`'s `[[links]]` and a raised `max_file_size_mb` are requests until granted with the new `code-index approve`. Built-in parsing has a wall-clock deadline. `read_code` serves indexed files only. Every git call is built in one place, with repository-configured programs neutralised. PR checkouts no longer persist the token. Both installers are https-only. - **#383:** `.code-index/` must be a real, user-owned directory with no links in it. State writes are durable. SQLite runs with `TRUSTED_SCHEMA=0`. A reset needs every schema object to be a known code-index name. A damaged index in its own state directory is reset in place. - **#381 / #382.3:** each watcher write is validated against disk, and `index_freshness` self-heals. Fixes a pathless-event panic that killed the watcher thread. The debouncer runs with NoCache (mass deletes are linear). - **#382 / #358 / #387:** - A daemon at its cap answers `busy` and never gets evicted. - A spawn poll stops early when the child it spawned has died. - `daemon.lock` is no longer unlinked. - `plugin rollback` and `plugin retry` exit non-zero under a live daemon. - An unchanged restart with the Vue package parses nothing. - `idle_timeout_min` is read from the config. - **#384:** a scoped pass works from a materialised ref set (rust-analyzer body edit 3.5 s → 1.0 s, binds identical on all 9 pinned repos). `quick_check` runs once per unchanged file. The graph tools walk by frontier. Schema 84: two redundant refs indexes dropped, `idx_symbol_edges_to` added. ## Integration fixes (beyond the lanes) - `idx_symbol_edges_to` added to `KNOWN_OBJECTS`. - The integrity check is skipped only for a file **unchanged since it passed** (creation time + mtime + length). P's path + ctime key let a database damaged in place after its check open unchecked, contradicting S1's reset test. Measured on a real daemon start: 1 check and 2 skips, so the saving holds. - `plugin retry` gets the same `BlockedByOwner` treatment as rollback, with a new test (mutation run: red). - W's pinned-link fixture grants its link. Lane D's two wall-clock tests run in `ci-timing`. ## Records - `host-package-lifecycle.json`: schema 83 → 84, conditions only (empty categorised diff). - `ruby-package-cost.json`: re-measured under schema 84, operator-approved after per-statement attribution against master. Package `vm_step` −204,248 (−0.73%): - `INSERT INTO refs`: −213,040 (two fewer index maintenances per row) - new reverse edge index: +12,517 Executions are identical statement by statement. `wall_ratio_pct` records the bless run's 141 (surveyed 134/139/141). - `baseline.json` and `cost-baseline.json` are untouched. ## Gate One full local run, including ci.yml's nightly-only jobs: everything green except four steps. - **Three were integration catches**, fixed above. clippy, docs, the workspace leg (5,148), the daemon leg (968) and the timing profile (36) were re-run green. - **`ovh_python`:** warm ratio 1.103 vs 1.10 at load 2–4.6. It was 1.03–1.04 on three isolated reruns. - `precision_gate`: 7/7, 0 phantoms. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
- #382.1: an over-cap connection is answered with an unsolicited `busy`
  frame (RpcError::BUSY, id 0) and closed (bounded: 16 concurrent answers,
  1 s each). The takeover probe counts `busy` as an answer, so a full cap
  never gets a healthy daemon evicted; RpcChannel treats it as a closed
  connection (heals the same way, says why).
- #382.2: both spawn polls (MCP attach, RpcIndex respawn) stop as soon as
  the child they spawned has exited with nobody else serving the root, and
  report its exit status and own exit record (restart::spawned_daemon_gone).
- #382.4: `plugin rollback` while a daemon owns the index exits non-zero
  (Performed::BlockedByOwner) and says nothing changed.
- #382.6: LockGuard::drop no longer unlinks daemon.lock while its lock is
  held (two owners on Unix, delete-pending open failures on Windows).
- #358: snapshot::load and the watcher's per-path lookup fold a file's
  several live contributions (fold_producer) instead of last-row-wins, so
  an unchanged restart with the Vue package parses 0 files (was
  parsed=4 reclassified=4). The daemon now reads
  `[daemon] idle_timeout_min` (flag wins; 0 = unset).
- #387: the idle-exit window proof reads the registration before a
  connect probe bounded at 250 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The project state directory lives inside the indexed tree, so a
repository can ship it, and an index database found there may not be
one this build wrote. Every state write and every destructive reset now
requires positive evidence of ownership.

- One state-directory check (code_index_core::state_fs::ensure_state_dir)
  at every first touch of `.code-index/` (index open, daemon lock, daemon
  log, workspace id, languages state): the directory must be a real
  directory owned by the effective user (unix) and contain no symbolic
  link or reparse point; otherwise the open is refused with the path and
  the repair.
- One durable write (state_fs::write_durable) for every state, approval
  and trust-store file: a new owner-only temporary (create_new, never an
  existing name), fsync, rename over the target, fsync the directory.
  The target is never deleted first, so a concurrent reader never sees
  it absent and a failed write leaves the old content (#382 item 5).
- SQLite is built with SQLITE_TRUSTED_SCHEMA=0 (.cargo/config.toml), so
  no connection trusts the schema of the database it opens; pinned by a
  test that reads the pragma off a bare connection.
- Index ownership is positive (#383): a reset is refused when any table,
  index, trigger or view falls outside the closed set of names some
  code-index schema created (KNOWN_OBJECTS, pinned against schema.sql),
  and the refusal is decided before the first write, so a refused file
  stays byte-identical.
- A damaged index (SQLITE_CORRUPT / NOTADB / quick_check) in its own
  state directory is reset in place (SQLITE_DBCONFIG_RESET_DATABASE +
  VACUUM) and opened again; elsewhere it is reported as before. doctor's
  integrity FAIL now names the remedy (#382 item 2).
- The newer-schema refusal names the database to delete
  (schema::recovery_hint_at), never the whole state directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
A committed .code-index.toml and the scanned working tree are written by
whoever wrote the repository. This closes the places where that input was
honoured as if the user had chosen it, with one mechanism per concern:

- Manifest grants: a [[links]] entry and an index.max_file_size_mb above the
  built-in default are REQUESTS. They take effect only once granted in the
  existing user-scoped approval record (new `manifest_grants` field, bound to
  the value requested). `code-index approve` lists and grants them;
  `code-index link add` grants the link it writes. The MCP reply
  (project_not_available, server instructions), `link list` and
  `code-index doctor` say when a request is pending and name the command.
  A repository may still lower the cap.
- Builtin parsing now has the package path's wall-clock deadline (same
  budget, same progress-callback mechanism, same transient refusal).
- read_code serves only files the index holds; anything else under the
  root is refused as path_not_indexed, naming the walker rule.
- Git: every invocation is built in one place. Filter drivers are
  enumerated across all repository config scopes with includes followed,
  and status/diff no longer recurse into submodules.
- CI: pull-request checkouts no longer persist the token in .git/config,
  and no remote URL carries it. install.sh downloads over https only
  (TLS 1.2+, no redirect downgrade with curl).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The writer re-validates every live message against disk as it applies it
(Writer::validating): a message about a file that is gone becomes a Delete
(no phantom row for a file renamed away before its insert committed); a
Delete for a file still there is skipped; a StatTouch that stamped nothing
and a Withdraw that removed live rows are applied and the path is handed
back. Handed-back paths return to the dispatcher (unbounded recheck
channel, sent after commit) and are decided again against the committed
state by the code that owns classification (recheck_path).

Self-heal: index_freshness queues every lagging path it measures through
the same queue (a whole-tree re-walk when the list is truncated), and
index_coverage's no_row_yet branch triggers that comparison, so its
"retry" hint is now true for a file lost in a race.

#382.3: the debouncer runs with NoCache. The file-id map only stitches
rename halves, which handle_event does not need; its per-remove scan was
quadratic (rm -rf drain 0.53/2.3/8.4 s at 2k/5k/10k files vs 0.25/0.30/
0.48 s). PathfulWatcher drops the pathless events notify's inotify backend
emits for forgotten watch descriptors, which panicked
notify-debouncer-full's paths[0] and killed the backend thread (seen in the
new storm harness).

RemovalProbe and the writer join index paths component by component
(disk_path), so a `\\?\` Windows root no longer makes every probe "gone".

Dogfood follow-ups: a reply pinned to one project probes partial sources of
that project only, so it no longer reads (and discloses index_state for)
primary; the lang hint says "nearest" when the whole spelling list fits.

Tests: watcher_convergence (deterministic storm under a held write lock,
live == cold on files/symbols/refs/binds), freshness_self_heal, writer and
watcher unit tests, index_coverage and pinned-reply e2e, go_ready wiring.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
- install.ps1: every remote download (asset and tag lookup) must be
  https://, except plain http to the loopback host; TLS 1.2+ is raised
  before each request. Same rule as install.sh.
- `code-index link remove` withdraws the manifest grant for that link, so
  a link later re-declared at the same name and path is a new request
  rather than pre-approved by a stale grant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follow-up to the state-directory trust change.

- Every client that reads the daemon registration does so through
  Lockfile::read_state, which now runs state_fs::check_state_dir (check,
  never create) first: a refused directory is an Err carrying the same
  refusal the writers give, so nothing it holds is attached to or
  polled. A directory another account owns keeps its #317 shape: its
  registration reads as Unreadable (classified by the lock), and an
  index open there keeps the named access-denied refusal.
- The project manifest writer (cli link) and the [languages] config
  writer go through state_fs::write_durable; the config is no longer
  deleted before the rename on Windows.
- The link scan inside the state directory skips entries that vanish
  mid-walk (a writer's temporary renamed away), which made a check
  racing a writer refuse a healthy directory.
- The refusal text says what removing the directory costs: a rebuild
  of the index and this checkout's stable handles. The other-account
  refusal says not to touch it and to use --db instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
`.code-index.toml` is repository content, not this tool's state. The
durable write now takes an Access: Private (owner-only, every state,
approval and trust-store file) or RepositoryFile (the existing file's
permission bits are kept; a new file gets the mode any file the user
creates gets). The manifest writer (cli link) and the [languages]
config writer use RepositoryFile; nothing else changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
SCOPED PASS. Its population (in a scoped file, or named like a symbol of
one) is materialised once as ref ids (`temp.resolve_pass_refs`) and every
statement is driven from it, as a durable batch's is. Candidate buckets
keep only the names of the pass's groups; relations keyed by the file of
the ref they serve (import_key_rel, local_bind, import_name_target,
python tainted names) are built for the pass's files only, unless
influence will classify the whole table. ref_count and symbol_edges are
maintained over the refs whose target the pass changed (plus the edited
files, plus an existence sweep for re-minted endpoints), not rebuilt.
MEASURED, pinned rust-analyzer, body edit of crates/hir/src/attrs.rs:
105.8M -> 35.1M vm_step; 3.47-3.60 s -> 1.02 s (3 alternating runs).
Bind/edge/ref_count projections identical to the prior binary's scoped
pass on all nine pinned repos (diff=0).

quick_check runs once per database file per process (path + creation
time); a daemon start opened its index three times (1.1 s each on RA).

Graph tools walk symbol_edges by frontier (new idx_symbol_edges_to)
instead of materialising the whole graph; size guard and stale_edges are
unchanged whole-table counts, so replies are identical.

resolution_gaps reads its candidate-pool join once.

Schema 84: drop idx_refs_name / idx_refs_file (prefixes of
idx_refs_name_lang / idx_refs_file_line), add idx_symbol_edges_to.
EXPLAIN QUERY PLAN over a traced drive of every RPC surface on RA
(1470 statements): 63 used a dropped index, all 63 now use the superset
index with an otherwise identical plan. One gated covering loss recorded
(epoch.rs table + COVERING_LOSSES). Cold cost: vm_step -0.6..-1.2% on the
seven priced repos, within the ratchet; baselines untouched.

Not re-recorded: tests/corpus/host-package-lifecycle.json needs a
conditions-only re-measure for schema 83 -> 84.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	crates/indexer/src/db.rs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
Same defect as rollback: retry writes no approval record and the daemon
never builds or promotes a generation on its own, so a deferred retry was
a zero exit over nothing done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
The once-per-file cache keyed on path + creation time skipped the check
for a database damaged in place after it passed, which is the case the
corrupt-reset path must catch. The identity is now creation time,
modification time and length, one passing identity per path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
ruby-package-cost: #384 schema 83 -> 84, re-measured and attributed per statement (operator-approved)
All checks were successful
Package CI / package (timeline) (pull_request) Successful in 29m28s
Package CI / package (csharp) (pull_request) Successful in 29m51s
Package CI / package (svelte) (pull_request) Successful in 30m8s
Package CI / package (vue) (pull_request) Successful in 30m15s
Package CI / package (xaml) (pull_request) Successful in 30m15s
Package CI / package (php) (pull_request) Successful in 30m36s
Package CI / package (rust) (pull_request) Successful in 30m43s
CI / CI lane wall-clock headroom (pull_request) Successful in 57s
Package CI / package (typescript) (pull_request) Successful in 31m22s
CI / guest crates (fmt, clippy, doc) (pull_request) Successful in 1m38s
Package CI / package (javascript) (pull_request) Successful in 32m13s
Package CI / package (ruby) (pull_request) Successful in 32m34s
Package CI / package (python) (pull_request) Successful in 37m34s
CI / cargo clippy (pull_request) Successful in 17m18s
CI / cargo deny (pull_request) Successful in 11m33s
CI / cargo test (abi, 32-bit + wasm32) (pull_request) Successful in 12m23s
CI / cargo check (MSRV 1.98) (pull_request) Successful in 14m0s
CI / cargo check (windows-gnu) (pull_request) Successful in 15m15s
Package CI / package-corpus (csharp) (pull_request) Successful in 48m1s
Package CI / package-corpus (python) (pull_request) Successful in 48m11s
Package CI / package-corpus (javascript) (pull_request) Successful in 48m13s
Package CI / package-corpus (rust) (pull_request) Successful in 48m8s
Package CI / package-corpus (php) (pull_request) Successful in 48m26s
Package CI / package-corpus (typescript) (pull_request) Successful in 45m33s
Package CI / package-corpus (ruby) (pull_request) Successful in 52m45s
Package CI / package overhead budgets (nightly, one package at a time) (pull_request) Has been skipped
CI / OSS corpus (tier 1) (pull_request) Successful in 46m55s
CI / cargo test (pull_request) Successful in 23m29s
CI / cargo test (daemon transport) (pull_request) Successful in 10m1s
CI / Plugin path cost + pool throughput (nightly) (pull_request) Has been skipped
f29a708601
package vm_step -204248 (-0.73%): INSERT INTO refs -213040 on both legs
(two fewer index maintenances per row), symbol_edges insert +12517 (the
new reverse index). Executions identical statement by statement; both
legs resolve 3804 refs. wall_ratio_pct records this bless run's reading
(141; surveyed 134/139/141 at load < 3).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
state_fs: the link walk reads the directory listing, not each path (Windows refused a racing check: access denied)
All checks were successful
Package CI / package (svelte) (pull_request) Successful in 29m57s
Package CI / package (timeline) (pull_request) Successful in 29m57s
Package CI / package (rust) (pull_request) Successful in 30m7s
Package CI / package (vue) (pull_request) Successful in 30m31s
Package CI / package (php) (pull_request) Successful in 30m56s
CI / CI lane wall-clock headroom (pull_request) Successful in 1m9s
Package CI / package (typescript) (pull_request) Successful in 31m33s
Package CI / package (csharp) (pull_request) Successful in 31m40s
Package CI / package (javascript) (pull_request) Successful in 31m54s
Package CI / package (ruby) (pull_request) Successful in 31m57s
CI / guest crates (fmt, clippy, doc) (pull_request) Successful in 1m48s
CI (Windows) / fmt + clippy + build + test (windows) (pull_request) Successful in 37m23s
CI / cargo check (MSRV 1.98) (pull_request) Successful in 19m9s
Package CI / package (python) (pull_request) Successful in 37m47s
CI / cargo doc (intra-doc links) (pull_request) Successful in 9m15s
CI / cargo deny (pull_request) Successful in 11m16s
CI / cargo test (abi, 32-bit + wasm32) (pull_request) Successful in 12m0s
CI / cargo check (windows-gnu) (pull_request) Successful in 14m51s
Package CI / package-corpus (csharp) (pull_request) Successful in 47m38s
Package CI / package-corpus (php) (pull_request) Successful in 47m47s
Package CI / package-corpus (javascript) (pull_request) Successful in 47m53s
Package CI / package-corpus (python) (pull_request) Successful in 47m54s
Package CI / package-corpus (rust) (pull_request) Successful in 48m4s
Package CI / package-corpus (typescript) (pull_request) Successful in 46m7s
Package CI / package-corpus (ruby) (pull_request) Successful in 52m31s
Package CI / package overhead budgets (nightly, one package at a time) (pull_request) Has been skipped
CI / cargo test (pull_request) Successful in 25m18s
CI / OSS corpus (tier 1) (pull_request) Successful in 36m19s
CI / cargo test (daemon transport) (pull_request) Successful in 10m4s
CI / Plugin path cost + pool throughput (nightly) (pull_request) Has been skipped
960f4704dc
On Windows a file deleted while a handle is still open is delete-pending,
and opening it by path fails ERROR_ACCESS_DENIED, not NotFound. CI run 1168:
a_check_racing_a_writer_never_refuses, 55 refusals in 5,000 checks.
DirEntry::metadata never follows a link; on Windows it is the listing's own
record (reparse attribute and tag included) and opens nothing, on unix it is
symlink_metadata. New Windows test: a junction inside the state dir is
refused. The racing test runs 50,000 checks: at 5,000 its mutation survived
3 runs of 10 on Linux.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDDBdWU4yThi8Zi51E6FCj
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
h-dv/code-index!388
No description provided.