Automate release tagging and make index builds resilient - #191

Summary

• Add Actions > Release > Run workflow on main with a required tag input. Validate the tag against the captured commit's workspace version, create it without force, and build/publish that exact commit in the same workflow. Preserve tag-push releases, matching lightweight/annotated-tag retries, per-tag concurrency, and publication-time revalidation. Peel captured annotated-tag object IDs to commits rather than resolving mutable tag names. Checkouts use trusted github.sha; downloaded archives remain separate from checked-out source when generating checksums.
• Make ordinary index and serve share serve.lock. Under that lock, recover journaled interrupted publications before cleaning recognized stale spill and staging artifacts. Preserve published files, unrelated entries, legacy unjournaled backups, and managed shared storage. Invalid/ambiguous journals, incomplete scratch enumeration, and linked/unremovable/invalid fixed staging entries stop startup before writers can reuse them. Spill cleanup can warn and defer because spill writers allocate fresh names. Windows-mapped files are not force-unlinked.
• Check destination free space before extraction and staged writes, with checked estimates for source bytes, output sizes, per-path overhead, and headroom. Do not invalidate an existing index when admission fails. Mark destination/spill I/O errors, including JSON output failures, so a cold build does not retry in heap when storage is unusable. Preserve native causes and managed commitment/retry state rather than classifying unrelated input failures as storage errors. Keep cold-start skeleton creation minimal so a rejected background build can expose its error through status and leave clients able to scan.
• Run initial, resumed, fallback, reload, and stale-delta builds in dedicated --max-cpu indexing pools. Do not change RAYON_NUM_THREADS or resize the global search pool. Pool-creation failures are surfaced explicitly.
• Save first-build progress through the existing external sorter and streaming publication path: after the first bounded extraction batch, approximately every 30 seconds between subsequent batches, and at completion. Resume persisted reader paths, reconcile edits/deletions/ignore changes before indexed queries, and recover process termination during publication. Read evidence alone cannot hide text missing from the reader.
• Keep rollback armed until publication commits. If rollback itself fails, preserve its journal and latch recovery-required state under the publication lock: later publishers, reloads, autosaves/checkpoints, filename-only saves, and bootstrap resets cannot overwrite unresolved recovery state. Status and reload diagnostics require a restart; existing in-memory readers remain available. Startup recovery completes before publication is admitted again.
• Harden owned-child cleanup and test clients. On Darwin, a zombie-only group can return EPERM; reap the owned child and require kernel-confirmed group disappearance within three seconds rather than ignoring permission errors. Persistent errors remain fatal. macOS descendant assertions use native process status, not absent procfs. Managed owner fixtures retry only explicitly retryable, uncommitted catalog contention, using the same claim and a ten-second deadline; a real SQLite writer-lock handshake covers that behavior.

The separate Azure DevOps Windows signing pipeline is unchanged. No release tag or public release has been created by this PR.

Operational limits

--max-cpu is a worker-count budget, not an OS CPU quota. External builds retain the 64 MiB posting buffer; --max-memory controls only the exceptional in-memory fallback. That fallback is prohibited after checkpoint publication is attempted or for destination, spill, and disk-space failures. Existing checkpoint progress is never discarded to enter it.

Checkpointing streams prior postings into replacements, adding disk I/O without materializing all postings in heap. The walk, one extraction batch, checkpoint publication, or final reconciliation can exceed the checkpoint interval. --no-watch retains full catch-up reads. Recovery covers process termination, not storage loss or power-failure durability. Free-space checks are estimates, not reservations or worst-case guarantees. No large-repository performance claim is made.

Validation

Final head 39e5d9ec3a68eca19a43411a5254b0a05f982bcd (tree 4757bb045ede898863549d49b9468a2fd2f02c57) passed the Windows/Linux/macOS CI matrix, including all three native managed lifecycle measurement jobs, and all four CodeQL analyses. The bounded CI measurements are not quiet-host or large-repository performance evidence. Copilot's review of this exact head reports 0 open findings, with no unresolved inline threads. It still recommends final human review; automated results do not replace the required approval.

The full local core and affected CLI qualification below was executed at production-changing commit 477120018e77bae2c958bcfac8f51f69f368ff15 (tree 4da07c882a5a885b159871a07bbee79661365d6d), before the test-only follow-up:

Ignored Rust entries are supervised subprocess helpers or the opt-in quiet-host measurement, not additional standalone passes. Counts exclude nested child-run summaries and focused reruns. The original tgrep 1.1.0 CLI and reader controls were used for compatibility checks and their hashes remained unchanged. Owned validation processes were cleaned up; the native source snapshot still matched all 208 archived files after qualification.

• cargo test --locked -p tgrep-core --features managed-test-hooks -- --test-threads=2
• cargo test --locked -p tgrep-cli --features managed-test-hooks --bin tgrep --test indexed_containment --test warm_start_gitignore --test ripgrep_compat --test shared_daemon -- --test-threads=2
• cargo fmt --all -- --check
• cargo clippy --locked --workspace --all-targets --features tgrep-cli/managed-test-hooks -- -D warnings
• python -B -m unittest discover -s scripts -p 'test_*.py' -v

All five command families passed on both platforms (python3 on Linux). Earlier full workspace qualification at 32fea82 passed 1,288 tests on Windows and 1,330 on native Linux, with six ignored entries per platform. Those earlier counts are not attributed to either later source.

The test-only follow-up 39e5d9e corrects the simulated-restart fixture after an Ubuntu CI writer-lock failure. It disables a nonexistent watcher so catch-up finishes synchronously, retains a duplicate writer handle as a deterministic control, proves that a live writer still excludes another writer, and explicitly unlocks only after fixture work is quiescent. All three rollback/restart tests failed with the duplicate-handle control before this correction. This commit changes only the test module; production logic remains that of 4771200.

At 39e5d9e, all CLI unit tests were rerun locally: 319 passed / 2 ignored on Windows, and 340 passed / 2 ignored on native Linux. Formatting and strict CLI/test Clippy also passed on both platforms, and all 208 native archived source files still matched. The full core/shared/compatibility suites were not rerun locally on this test-only follow-up; the final exact-head CI linked above supplies the full native matrix.

• cargo test --locked -p tgrep-cli --features managed-test-hooks --bin tgrep -- --test-threads=2
• cargo fmt --all -- --check
• cargo clippy --locked -p tgrep-cli --features managed-test-hooks --bin tgrep --tests -- -D warnings

Coverage includes an owned server terminated after a 1,024-file checkpoint and restarted to serve real indexed RPC results; scan fallback while incomplete; indexing/search pool isolation; failed-checkpoint resume; missing reader entries despite existing evidence; empty/short/binary-only inputs; recovery at each file-move boundary; malformed/ambiguous journals; failed-commit rollback; low-space/probe failures; mapped-file cleanup retry; and real CLI writer commands refusing linked staging without touching external sentinel files.

Failed-rollback regressions exercise move, reader-validation, and commit failures through both publication paths, then reject all subsequent publishers, preserve the old reader/journal, and recover through restart. Successful rollback still permits reload. These regressions reproduced unsafe later publication before the fix. The owner-claim fixture also rejects nonretryable, committed, unknown-commit, unrelated-lock, and expired retry cases.

Storage-error regressions cover real spill-directory/segment creation and spill-read failures; every core output filename with both sort strategies; native write failures and JSON-wrapped I/O; unchanged managed error envelopes and non-I/O failures; destination versus input-path provenance; and cold-server failure/recovery before its first checkpoint. Three core regressions and the cold-server regression failed before the fix, with the latter actually entering the heap fallback; all now pass on both platforms.

All 17 real-Git release-helper cases pass on both platforms. They cover raw annotated-tag object SHAs in shallow/no-tags checkouts for preparation and publication verification, exact snapshot matching, invalid object inputs, retries, conflicting/deleted tags, and read/push failures. Annotated-tag and linked-staging regressions also failed before their fixes. Deterministic teardown tests cover transient Darwin permission errors, confirmed disappearance, persistent denial, and unchanged non-Darwin error handling; the actual macOS teardown tests passed in the final exact-head CI. These are bounded regression fixtures, not large-repository benchmarks. The unchanged workflow previously passed actionlint 1.7.12; public tag creation and release publication are deliberately not exercised before merge.