Skip to content

perf(capture): reject bounded writes before loading embeddings - #492

Merged
cdeust merged 4 commits into
mainfrom
perf/green-exact-preflight
Sep 8, 2026
Merged

cdeust merged 4 commits into
mainfrom
perf/green-exact-preflight

Conversation

@cdeust

@cdeust cdeust commented Sep 7, 2026 •

Copy link
Copy Markdown
Owner

Reject captures whose existing novelty bound proves they cannot pass, before loading the embedding model. Admitted captures retain the normal write gate.

Shared unrounded observations and preflight branches are covered by handler/core tests.

Validation and review (2026-09-08):

  • Reviewed the implemented diff for correctness, security, lifecycle and ordering; no unresolved source finding.
  • Integrated current main (including deps: refresh compatible dependencies and align plugin launcher pins #491) and every updated parent. Reviewed head: 257513c0ae6d96e96d8b9e37267cb5b7e0d72b45.
  • Combined runtime tree: locked Ruff/format, craftsmanship and Pyright pass; full suite 8,013 passed, 257 skipped, one stale private dependency-fixture failure. Correcting that fixture without changing assertions yields 4/4 launcher tests passing. The added baseline setup and matrix tests pass 8/8. GitHub CI is running on each new head.
  • Full LongMemEval/LoCoMo comparison against main ca32ba50 is running on separate disposable databases with identical public datasets. It uses the repository's existing 0.005 tolerance; no acceptance result is claimed before completion.

Historical measurements apply to their recorded revisions and scopes. The current PR is authorized for merge after applicable checks; remaining optimization targets above are not presented as completed benefits.

Brings in #503 (floor check non-blocking + --no-regression gate) and
#505 (mcp-host-config timeout sized to its measured tail), so this PR
is verified against the CI its merge will actually land on.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01StMBvNd7eVJGtpnC2zNsx1
Brings in #506 (docker-smoke cache export skipped on pull_request), the
fix for the Docker Smoke timeout that was the only red check on this
branch's last run.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01StMBvNd7eVJGtpnC2zNsx1
pull Bot pushed a commit to asleekgeek/Cortex that referenced this pull request Sep 8, 2026
main itself no longer clears the published FLOOR_* thresholds in
reproduce.sh (LongMemEval MRR 0.904988 < 0.914, LoCoMo MRR 3-run mean
0.779868 < 0.805, LoCoMo Recall@10 0.889506 < 0.915 — all measured via
this same script, cdeust/Cortex PR cdeust#492), so every PR built on main
inherited an already-failing gate it did not cause and could not pass.

check_floors now rounds `got` to the same 4-decimal precision it prints
before computing delta/status (comparing at a hidden precision beyond
what's displayed was the source of borderline-case confusion), and no
longer exits 1 — same disposition as the pyright ratchet in issue cdeust#188
("raising the floor would just hide the debt").

Add benchmarks/lib/bench_regression.sh + reproduce.sh --no-regression:
runs the same suite against --baseline-ref (default origin/main) in an
isolated git worktree, in the same container, then fails only if HEAD
is worse than its own baseline by more than the tolerance. This is the
blocking gate a PR should use instead of lowering FLOOR_*.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01StMBvNd7eVJGtpnC2zNsx1
pull Bot pushed a commit to asleekgeek/Cortex that referenced this pull request Sep 8, 2026
The Docker Smoke job wrote its buildx layer cache with
`cache-to: type=gha,mode=max` on every event. On run 34225008423
(PR cdeust#492) the image build finished at 70.8s and the job was still in
`exporting to GitHub Actions Cache` when the 13-minute bound fired at
827s. The smoke-test body never ran, so the PR went red on a step that
asserts nothing about the code under review.

GitHub Actions scopes a cache entry to the ref that created it: the
default branch's entries are readable from every branch, but an entry a
PR run writes is readable only by that same PR's later runs. This job
runs once per head commit, so a PR-written entry is exported and then
never read — pure cost on the critical path.

Skip the export on `pull_request`. `cache-from` stays unconditional, so
PRs keep reading main's warm cache and the ~200MB torch CPU-wheel layer
is still not re-downloaded; on run 34217523633 (push to main, warm
cache) the whole job took 242s with the export included.

The `timeout-minutes: 13` comment cited "run 33951959734, max 369s" as
its basis; today's 827s observation invalidates that as an upper bound.
It is re-sourced to the warm-path steady state it actually bounds, and
records explicitly that the cold `mode=max` export is removed rather
than budgeted for.

The two sibling Docker Build jobs already used `mode=min`; this job was
the odd one out.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01StMBvNd7eVJGtpnC2zNsx1
@cdeust
cdeust marked this pull request as ready for review September 8, 2026 19:37

@cdeust cdeust left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Completed source review of this implemented scope and its parent integration. No unresolved correctness, security or lifecycle finding. The description records limitations rather than claiming unmet optimization targets. Merge remains gated on current required CI and applicable baseline quality evidence. Review includes independent read-only review of the per-PR diff and combined locked local tests.

@cdeust
cdeust merged commit ac28344 into main Sep 8, 2026
28 checks passed
@cdeust
cdeust deleted the perf/green-exact-preflight branch September 9, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants