Skip to content

fix(exec): derive the query memory budget from the process limit and let joins spill or re-side instead of failing at a fixed 512 MiB #1985

Closed

Description

@DecisionNerd

Part of #1982.

Problem

Query memory is a fixed 512 MiB DataFusion pool (SessionResourceConfig::default, crates/graphforge-exec/src/session.rs) with spill disabled by default, and hash joins cannot spill at all. Two failures at SF1 on frozen c09577e2e:

A read that a 4 GiB envelope could afford fails because of an arbitrary constant, and because the build side of a join has no path to disk.

Expected

  • The query budget derives from the process's memory limit (cgroup memory.max, then host memory) with a stated fraction and floor, the way the bulk builder already reads its cgroup budget; an explicit GraphForgeOptions budget still overrides it.
  • Operators that can spill (sort, aggregate) spill to the configured spill directory under the existing cap when the budget is exceeded, rather than failing.
  • Hash joins either build the smaller side (planner chooses build side by estimated cardinality, with the anchored side as the probe) or partition the build side so each partition fits; a join that still cannot fit returns the typed GF_EXECUTION resource error it returns today, with the consumers named.

Acceptance criteria

  • IC3's three pinned bindings complete on the SF1 SNB Interactive project under the 4 GiB envelope with correct results; latencies posted.
  • A facade test runs a join whose build side exceeds a deliberately small budget and either spills or re-sides, with the result unchanged, and the pool's peak reservation asserted below the budget.
  • Budget derivation is covered by a test with a fake cgroup memory.max.
  • No test or production code sets a budget just to make a test pass.

Activity

  1. added
    bugSomething isn't working
    coreCore source code changes
    executorChanges to query executor
    on Oct 12, 2026
  2. DecisionNerd commented on Oct 12, 2026

    @DecisionNerd
    ContributorAuthor

    Widened by the probe snapshot on #1982 (2026-10-12): the fixed pool also fails aggregates, not only joins. IC6 fails Resources exhausted … GroupedHashAggregateStream, IC9 fails … AggregateStream, IC7 fails HashJoinInput. Aggregates can spill in DataFusion but spill is disabled by default (spill_enabled: false), so they fail at 512 MiB too. Scope here covers both: derive the budget from the process limit, enable bounded spill for sort and aggregate, and re-side or partition hash joins. Acceptance adds IC6, IC7 and IC9 pinned bindings completing under the 4 GiB envelope with correct results.

  3. 44 remaining items

  4. DecisionNerd commented on Oct 13, 2026

    @DecisionNerd
    ContributorAuthor

    U4 status (PR #2049, perf/1985-anchored-varlen-build-estimate, head e6b4b4b).

    Fix: a current persisted equality index now bounds PropertyOverlayExec's published row count by the manifest shards that can hold the value, so an anchored seed is no longer estimated as 20% of the route (IS6 traced at 636k seed rows, truth 1, join-left estimate 1,420,322,040 B against the 1 GiB limit under a 4G scope). The flip threshold and expand_statistics.rs are unchanged.

    Verified: graphforge-exec 1131/1131 passed; the neighbouring graphforge-api suites (including query_memory_budget, traversal_join_memory, enrichment_frontier_reads, variable_length_frontier_reads, partitioned_expand, fixed_hop_limit) 91/91 passed, 2 skipped (release-only ignored benches); bdd 118/118 and TCK 3898/3898 with 0 regressed; make check exit 0. The new facade test fails on revert at the plan assertion (SortMergeJoinExec plus an unpruned enrichment).

    NOT measured: IS6 / IS2 / IC2 / IC9 under MemoryMax=4G on the SF1 IC copy. I stopped before the release builds because df / fell below the 40 GB floor (24 GB free, 98% used, during the run) and the host load was 31. The p50 < 1 s target is therefore unconfirmed. When measuring, rebuild the property indexes on the copy with the same build as the binary under test (#2044 changed the pack format), and build the "before" binary from the same tree with property_scan.rs and property_equality_index.rs at their main versions.

  5. DecisionNerd commented on Oct 13, 2026

    @DecisionNerd
    ContributorAuthor

    PR #2050 (Part of #1985) fixes the TCR2 CPU regression from #2020: the partitions queued on the shared pool mutex through per-path-state try_grow/shrink, then on the decoded CSR shard cache mutex.

    TCR2 SF1, bindings b0-b2 combined, median of 3 rounds (CPU-seconds, systemd-run --scope -p MemoryMax=4G)

    arm run CPU sys wall host load1
    base-main (d2f07c2) ab6 437.8 218.1 67.8 8-12
    chunked reservation only ab6 78.9 28.6 17.4 8-12
    pre-#2020 emulation ab6 49.6 10.6 27.6 8-12
    chunked reservation only ab7 71.2 18.8 20.4 11-18
    chunked + shard-cache RwLock (PR head) ab7 57.7 12.0 16.9 14-21
    pre-#2020 emulation ab7 57.2 12.1 31.1 11-16

    Regression checks (median CPU-s): TCR1 2.6 -> 2.8 (ab6), 3.1 -> 3.1 (ab8); TCR8 14.5 -> 14.7 (ab6), 16.7 -> 16.1 (ab8, load 22-28). All bindings returned the reference row counts.

    Raw per-round results (ab6, ab7, ab8)

    ab6

    query arm round cpu_user cpu_sys cpu wall maxrss MB load before load after bindings ok
    TCR1 base-main 0 2.2 0.4 2.5 1.5 229 8.54/13.26/14.12 8.54/13.26/14.12 True
    TCR1 base-main 1 2.2 0.4 2.6 1.1 231 10.61/12.17/13.57 10.61/12.17/13.57 True
    TCR1 base-main 2 2.6 0.5 3.1 2.3 228 20.66/14.24/14.03 20.66/14.24/14.03 True
    TCR1 branch-main 0 2.4 0.4 2.8 1.8 225 8.54/13.26/14.12 8.54/13.26/14.12 True
    TCR1 branch-main 1 2.3 0.4 2.7 1.2 230 10.84/12.24/13.60 10.84/12.24/13.60 True
    TCR1 branch-main 2 2.6 0.5 3.1 2.6 229 20.66/14.24/14.03 20.60/14.34/14.06 True
    TCR1 pre2020-main 0 2.3 0.4 2.6 1.4 225 8.54/13.26/14.12 8.34/13.14/14.07 True
    TCR1 pre2020-main 1 2.2 0.4 2.6 1.2 231 10.84/12.24/13.60 10.61/12.17/13.57 True
    TCR1 pre2020-main 2 2.6 0.5 3.1 2.3 230 20.45/14.09/13.98 20.66/14.24/14.03 True
    TCR2 base-main 0 227.5 230.1 457.6 70.1 1294 11.64/15.24/14.75 12.45/14.77/14.63 True
    TCR2 base-main 1 219.8 218.1 437.8 67.8 1255 7.99/12.34/13.76 11.26/12.41/13.68 True
    TCR2 base-main 2 212.1 190.6 402.7 61.9 1263 9.63/11.81/13.41 11.77/12.01/13.37 True
    TCR2 branch-main 0 48.1 28.6 76.8 17.2 1132 12.45/14.77/14.63 12.14/14.59/14.57 True
    TCR2 branch-main 1 50.0 33.2 83.2 17.4 1184 8.34/13.14/14.07 8.06/12.83/13.96 True
    TCR2 branch-main 2 51.7 27.2 78.9 19.2 1158 11.77/12.01/13.37 11.37/11.91/13.31 True
    TCR2 pre2020-main 0 39.5 10.1 49.6 28.0 2281 12.14/14.59/14.57 10.13/13.86/14.33 True
    TCR2 pre2020-main 1 39.7 10.6 50.2 27.6 2307 8.06/12.83/13.96 7.99/12.34/13.76 True
    TCR2 pre2020-main 2 38.7 10.9 49.6 27.0 2183 10.61/12.17/13.57 9.63/11.81/13.41 True
    TCR8 base-main 0 11.5 1.4 12.9 5.4 461 10.13/13.86/14.33 9.64/13.70/14.27 True
    TCR8 base-main 1 13.2 1.4 14.5 5.7 472 11.00/12.29/13.62 10.84/12.24/13.60 True
    TCR8 base-main 2 14.5 1.5 16.0 12.5 459 15.85/12.82/13.58 19.21/13.64/13.83 True
    TCR8 branch-main 0 11.2 1.2 12.5 5.2 465 9.64/13.70/14.27 9.18/13.54/14.21 True
    TCR8 branch-main 1 13.1 1.6 14.7 6.1 436 11.26/12.41/13.68 11.00/12.34/13.65 True
    TCR8 branch-main 2 14.8 1.5 16.3 11.3 467 19.21/13.64/13.83 20.45/14.09/13.98 True
    TCR8 pre2020-main 0 11.1 1.3 12.3 7.2 466 9.18/13.54/14.21 8.54/13.26/14.12 True
    TCR8 pre2020-main 1 12.4 1.3 13.7 8.2 422 11.00/12.34/13.65 11.00/12.29/13.62 True
    TCR8 pre2020-main 2 14.4 1.6 16.0 15.3 505 11.37/11.91/13.31 15.85/12.82/13.58 True

    ab7

    query arm round cpu_user cpu_sys cpu wall maxrss MB load before load after bindings ok
    TCR2 branch-main 0 52.4 18.8 71.2 20.4 1179 17.87/21.56/19.19 17.08/21.13/19.10 True
    TCR2 branch-main 1 50.0 23.7 73.7 17.7 1165 11.96/18.51/18.37 11.31/18.04/18.21 True
    TCR2 branch-main 2 49.4 13.2 62.6 22.4 1143 14.88/18.07/18.21 17.86/18.57/18.38 True
    TCR2 branch3-main 0 45.7 12.0 57.7 16.9 1078 17.08/21.13/19.10 16.44/20.80/19.02 True
    TCR2 branch3-main 1 43.8 11.6 55.4 15.0 1134 14.62/19.93/18.79 14.19/19.57/18.69 True
    TCR2 branch3-main 2 49.5 12.1 61.6 25.1 994 17.86/18.57/18.38 20.69/19.17/18.58 True
    TCR2 pre2020-main 0 45.0 12.1 57.2 31.1 1944 16.44/20.80/19.02 14.62/19.93/18.79 True
    TCR2 pre2020-main 1 39.5 10.8 50.3 27.4 2118 14.19/19.57/18.69 11.96/18.51/18.37 True
    TCR2 pre2020-main 2 52.9 13.0 65.9 36.9 2183 11.31/18.04/18.21 14.88/18.07/18.21 True

    ab8

    query arm round cpu_user cpu_sys cpu wall maxrss MB load before load after bindings ok
    TCR1 base-main 0 2.6 0.5 3.1 2.0 223 24.14/20.42/19.05 24.14/20.42/19.05 True
    TCR1 base-main 1 2.6 0.5 3.1 2.3 232 26.86/21.35/19.39 28.15/21.71/19.52 True
    TCR1 branch3-main 0 2.6 0.5 3.1 1.9 224 24.14/20.42/19.05 23.80/20.41/19.05 True
    TCR1 branch3-main 1 2.6 0.5 3.1 2.7 231 26.86/21.35/19.39 26.86/21.35/19.39 True
    TCR8 base-main 0 15.2 1.8 17.0 10.5 456 22.28/19.82/18.82 23.49/20.17/18.95 True
    TCR8 base-main 1 14.9 1.5 16.4 10.0 458 25.96/20.99/19.25 26.86/21.35/19.39 True
    TCR8 branch3-main 0 14.6 1.5 16.1 8.9 469 23.49/20.17/18.95 24.14/20.42/19.05 True
    TCR8 branch3-main 1 14.6 1.6 16.1 9.1 453 23.80/20.41/19.05 25.96/20.99/19.25 True
  6. DecisionNerd commented on Oct 13, 2026

    @DecisionNerd
    ContributorAuthor

    U4 measurement on a real SF1 IC copy (PR #2049, head e6b4b4b)

    Result: real IS6 no longer flips at the 4G-derived budget. p50 3.74 s -> 0.22 s under MemoryMax=4G. IS2, IC2 and IC9 are unchanged.

    Method

    • Two fresh cp -a copies of the SF1 IC project (targeted chmod u+w, originals untouched), one per build.
    • After = release probe at e6b4b4b8d (based on d2f07c2b7, includes perf(storage): write edge-property pack columns as small authenticated pages (Part of #1984) #2044). Before = the same tree with only property_scan.rs and property_equality_index.rs checked out from origin/main (identical to d2f07c2b7 for those files).
    • rebuild_property_indexes was run on each copy by the same build as the binary under test (perf(storage): write edge-property pack columns as small authenticated pages (Part of #1984) #2044 changed the pack format): 778 equality files / 77,560,132 B and 40 edge-property files / 159,919,725 B on both copies.
    • systemd-run --user --scope -q -p MemoryMax=4G -p MemorySwapMax=0, run_suite.py bindings 0-2, two rounds interleaved before/after (round 2 reversed order), so 6 samples per row. Wall is per binding; CPU is user+sys of the whole probe process (3 bindings plus the ~0.35 s project open) from /usr/bin/time.
    • CONTENDED: load average was 12-21 on 16 cores throughout (shared host), so absolute times carry load noise. The before/after pairs ran back to back.
    Query Build Wall p50 (s) Wall range (s) CPU r1 / r2 (s) Rows b0,b1,b2 Load
    IS6 before 3.741 2.982-5.528 14.1 / 11.4 1,1,1 13.7-21.2
    IS6 after 0.223 0.181-0.763 1.3 / 1.1 1,1,1 13.7-19.9
    IS2 before 0.791 0.652-0.904 3.1 / 2.5 6,10,10 13.9-19.9
    IS2 after 0.799 0.667-0.915 3.0 / 2.7 6,10,10 13.9-19.3
    IC2 before 1.991 1.634-2.148 9.2 / 8.4 20,20,20 13.1-19.3
    IC2 after 2.097 1.807-2.242 6.7 / 6.7 20,20,20 13.6-18.3
    IC9 before 18.961 17.265-20.969 149.3 / 146.6 20,20,20 12.2-17.8
    IC9 after 18.327 17.546-19.989 143.8 / 147.6 20,20,20 12.9-14.3

    Rows: identical between builds for every binding. Against the earlier reference run (gdc-diag-1914b/results/ic*.json), IC2 (20,20,20) and IS2 (6,10,10) match. IS6 returns 1 row per binding (one container forum and moderator per message). IC9 errored in that old reference run, so its 20 rows are compared before vs after only.

    Plan check (explainparam, IS6 bindings 0-2, run inside the same 4G scope): before has 2 SortMergeJoinExec (both Left enrichment joins, on=[(node_uuid, node_uuid)]); after has 0 SortMergeJoinExec and 8 HashJoinExec on every binding. So the sort-merge left-enrichment pruning noted in the issue is not needed for this shape and is not added.

    Still open and not part of this change: IS2 (0.8 s) and IC2 (2.0 s) are under contended load and unchanged by the PR; IC9 (18 s, 145 s CPU) is unchanged and belongs to its own unit.

  7. DecisionNerd commented on Oct 14, 2026

    @DecisionNerd
    ContributorAuthor

    Fresh SF1 IC3 proof at fbf5a3c63 exposed one remaining join reservation defect, so this PR stays draft despite green exact-head CI. All three pinned bindings were attempted once after an excluded warmup: p000 took 3.199 s (1 row), p002 took 2.742 s (0 rows), both exact reference/digest matches; p001 returned GF_EXECUTION when its empty external sorter requested 35.3 MB against 34.0 MB heuristic quota. The actual pool held 544.4 MB of a 2 GiB budget.

    The native process-tree highwater was 2,523,705,344 bytes under observed memory.max=4294967296, with no OOM or process swap. Catalog counts reconciled. This is 2 matched / 3 attempted, 1 failed, 0 unrun, not a passing variant median or full coverage claim. No failed binding was retried. Raw driver/results/reference/native evidence and command/hash manifest preserve the failure.

    The cause is verified in DataFusion: an empty sorter cannot spill, so refusal of its first batch is fatal. Two new deterministic regressions reproduce that refusal before the repair, including an actual external sort. The bounded repair will apply producer headroom once a sorter has buffered input; its first batch still must fit the real pool and aggregate floors. No budget, timeout, reference, or assertion is being relaxed. A changed-source validation and fresh CI will be required before merge.

  8. DecisionNerd commented on Oct 14, 2026

    @DecisionNerd
    ContributorAuthor

    Fresh changed-source IC3 validation passed at d77d09f895a89d9bbe4a2413e237f28f678089df. All three pinned SF1 bindings completed once after an excluded warmup, under the observed native 4 GiB process-tree limit and 120 s per-binding timeout.

    Binding Driver latency Rows Reference and emitted digest
    p000 3.190587422 s 1 Exact match
    p001 2.989806018 s 0 Exact match
    p002 2.630509510 s 0 Exact match

    Nearest-rank p50 2.989806018 s, p95 3.190587422 s. 3 checked / 3 matched, 0 unchecked, 0 failures or unrun selected bindings. Driver/checker exit 0; total/per-label/per-type counts reconciled. Native highwater 2,559,234,048 bytes; no OOM or process swap. The reference is the harness's spec-derived IC reference; this is IC3-only diagnostic proof (3 of the full Interactive suite's 600 bindings), not audited certification or the parent-suite gate.

    Raw results, reference, native recordings, exact commands and all input/output/executable hashes. The previous FBF first-buffer failure remains intact; no failed binding was retried on unchanged source. Two new regressions fail against that old rule and pass with the bounded repair; the real sort verifies 262,144 exact sorted values, actual spilling, the original real cap and reservation cleanup.

    All 58 focused runtime/golden/facade tests passed, including the original 128 MiB join tests and 16 MiB extremum controls. Fake-cgroup budget derivation remains covered by the existing resource-policy tests. Final make check, formatting and diff checks passed. Independent final review is clear; CodeRabbit's two-file repair review completed with zero findings. Current-head CI and fresh GitHub reviews remain pending; PR #2115 is ready for review and will merge only after the required gate and review resolution.

  9. DecisionNerd commented on Oct 14, 2026

    @DecisionNerd
    ContributorAuthor

    Final current-head IC3 proof passed at c6bb3c5f6db3a31991f513239375a3edc8dd8b8e: 3/3 pinned bindings completed and matched the exact spec-derived harness reference and emitted digests, with the excluded warmup completed. Driver latencies p000/p001/p002: 3.174785956 / 2.968538840 / 2.580869662 seconds; nearest-rank p50 2.968538840 s. Native process-tree highwater 2,647,412,736 bytes, actual 4 GiB memory limit, no OOM or process swap, per-binding timeout 120 s. Driver and reference checker exited 0; catalog counts reconciled. This is IC3-only diagnostic proof (3/600 Interactive bindings), not the full parent baseline or audited certification.

    All 59 focused runtime/golden/facade tests, final make check, formatting and diff checks passed. The verified Greptile view-sizing finding is fixed and its thread resolved; independent current-source review is clear and CodeRabbit's two-file review completed with zero findings. Exact-head CI Gate passed with all product lanes green. PR #2115 is CLEAN and closes exactly this issue. It is ready for the squash merge queue.

    The earlier failed FBF diagnostic and D77 passing diagnostic remain preserved. No failed binding was retried on unchanged source; this new validation follows the reviewed compact-window repair.

    Final C6 raw driver, input/executable hashes, native evidence and emitted result JSON

    ic3-evidence.json

    {
      "schema": "graphforge-gdc-query-evidence/1",
      "certification": false,
      "suite": "snb-interactive",
      "status": "passed",
      "query_timeout_seconds": 120,
      "failures": [],
      "driver": {
        "name": "graphforge-benchmark-gdc-scorecard query",
        "version": "0.0.0",
        "executable_sha256": "2cfcdaa6cccc2df5397507fc127d9cf16471c5050145e8fb8bfc671f500e59da"
      },
      "inputs": {
        "workload_sha256": "d0ad62c9ef96260bdb4ed224e0c5ba0b75ad9905cc12e7b6f764be1b4cd0ef57",
        "expected_counts_sha256": "9daa4440bfbac2f7d3b8165445f4afab67890cb71267d85a7691680269926feb"
      },
      "project": {
        "path": "/home/ubuntu/gf-beta-evidence/join-candidate-view-window-fixed/ic",
        "opened_with": "graphforge_api::GraphForge::new(Some(path))"
      },
      "reconciliation": {
        "status": "reconciled",
        "source": "https://github.com/ldbc/ldbc_snb_docs/blob/b2269610f433da72e7c97041f01680aae369a903/tables/legacy/table-number-of-entities-interactive.tex (sf1; every table reconciles to the listed records of the pinned archive)",
        "method": "Cypher count probes through GraphForge::execute, plus GraphForge::labels",
        "nodes": {
          "expected": 3181724,
          "observed": 3181724
        },
        "edges": {
          "expected": 17256038,
          "observed": 17256038
        },
        "labels": {
          "City": {
            "expected": 1343,
            "observed": 1343
          },
          "Comment": {
            "expected": 2052169,
            "observed": 2052169
          },
          "Company": {
            "expected": 1575,
            "observed": 1575
          },
          "Continent": {
            "expected": 6,
            "observed": 6
          },
          "Country": {
            "expected": 111,
            "observed": 111
          },
          "Forum": {
            "expected": 90492,
            "observed": 90492
          },
          "Person": {
            "expected": 9892,
            "observed": 9892
          },
          "Post": {
            "expected": 1003605,
            "observed": 1003605
          },
          "Tag": {
            "expected": 16080,
            "observed": 16080
          },
          "TagClass": {
            "expected": 71,
            "observed": 71
          },
          "University": {
            "expected": 6380,
            "observed": 6380
          }
        },
        "types": {
          "CONTAINER_OF": {
            "expected": 1003605,
            "observed": 1003605
          },
          "HAS_CREATOR": {
            "expected": 3055774,
            "observed": 3055774
          },
          "HAS_INTEREST": {
            "expected": 229166,
            "observed": 229166
          },
          "HAS_MEMBER": {
            "expected": 1611869,
            "observed": 1611869
          },
          "HAS_MODERATOR": {
            "expected": 90492,
            "observed": 90492
          },
          "HAS_TAG": {
            "expected": 3721417,
            "observed": 3721417
          },
          "HAS_TYPE": {
            "expected": 16080,
            "observed": 16080
          },
          "IS_LOCATED_IN": {
            "expected": 3073621,
            "observed": 3073621
          },
          "IS_PART_OF": {
            "expected": 1454,
            "observed": 1454
          },
          "IS_SUBCLASS_OF": {
            "expected": 70,
            "observed": 70
          },
          "KNOWS": {
            "expected": 180623,
            "observed": 180623
          },
          "LIKES": {
            "expected": 2190095,
            "observed": 2190095
          },
          "REPLY_OF": {
            "expected": 2052169,
            "observed": 2052169
          },
          "STUDY_AT": {
            "expected": 7949,
            "observed": 7949
          },
          "WORK_AT": {
            "expected": 21654,
            "observed": 21654
          }
        }
      },
      "latency_clock": {
        "id": "graphforge-gdc-query-clock/1",
        "producer": "graphforge-benchmark-gdc-scorecard query",
        "source": "std::time::Instant",
        "unit": "ns",
        "interval": "one public GraphForge API call returning a fully materialized Arrow result",
        "warmup_passes_per_variant": 1,
        "percentile_method": "nearest_rank"
      },
      "result_digest": "graphforge-gdc-result-digest/1",
      "variants": [
        {
          "query_id": "IC3",
          "interface": "graphforge_api::GraphForge::execute_with_params",
          "ordered": true,
          "columns": [
            "friendId",
            "friendFirstName",
            "friendLastName",
            "xCount",
            "yCount",
            "xyCount"
          ],
          "status": "measured",
          "warmup": {
            "binding_id": "p000",
            "excluded": true,
            "completed": true
          },
          "samples": [
            {
              "binding_id": "p000",
              "status": "measured",
              "latency_ns": 3174785956,
              "rows": 1,
              "result_sha256": "39fe4a987576dc1620e3aaad2b700fbdadc77f5cc67e9523e38bbe251afadf69"
            },
            {
              "binding_id": "p001",
              "status": "measured",
              "latency_ns": 2968538840,
              "rows": 0,
              "result_sha256": "0c50a11692c6f222c4fa3c454f546946bd4ac403cd1be936d74e18b2859cbfcd"
            },
            {
              "binding_id": "p002",
              "status": "measured",
              "latency_ns": 2580869662,
              "rows": 0,
              "result_sha256": "0c50a11692c6f222c4fa3c454f546946bd4ac403cd1be936d74e18b2859cbfcd"
            }
          ],
          "summary": {
            "count": 3,
            "p50_ns": 2968538840,
            "p95_ns": 3174785956
          }
        }
      ]
    }

    ic3-checked-report.json

    {
      "scope": "diagnostic; does not certify full pinned coverage",
      "source_sha": "c6bb3c5f6db3a31991f513239375a3edc8dd8b8e",
      "driver_sha256": "2cfcdaa6cccc2df5397507fc127d9cf16471c5050145e8fb8bfc671f500e59da",
      "driver_evidence_sha256": "99dac60a5cdc260e69779b5b47ed73edb3499b98ef53e4fcd646476d2eaf75c5",
      "native_ok": true,
      "native": {
        "starttime": "2026-10-14 17:24:11.763720+00:00",
        "walltime": 19.253061415976845,
        "cputime": 59.779501,
        "memory": 2647412736,
        "blkio-read": 807706624,
        "blkio-write": 3290451968,
        "pressure-cpu-some": 0.098013,
        "pressure-memory-some": 0.0,
        "pressure-io-some": 0.174139,
        "exitcode": "return value 0",
        "exit_value": 0,
        "exit_signal": null
      },
      "variants": [
        {
          "query_id": "IC3",
          "status": "measured",
          "samples": 3,
          "measured": 3,
          "failures": [],
          "latencies_seconds": [
            3.174785956,
            2.96853884,
            2.580869662
          ],
          "p50_seconds": 2.96853884,
          "warmup": {
            "binding_id": "p000",
            "excluded": true,
            "completed": true
          }
        }
      ],
      "correctness": {
        "schema": "graphforge-gdc-rung-correctness/1",
        "status": "passed",
        "reference_source": "a spec-derived reference (gdc_snb_interactive_reference over the loaded snapshot), not the LDBC validation set (the pinned validation stream begins with an update at position 0)",
        "reference_sha256": "723cde5975f11f290fc6ad342b8dfdb7172a3cc18b85de1200ebcc5cd3d7010f",
        "checked": 3,
        "matched": 3,
        "unchecked": 0,
        "queries": {
          "IC3": {
            "matching": "exact",
            "checked": 3,
            "matched": 3
          }
        },
        "mismatches": []
      },
      "all_selected_bindings_completed": true,
      "all_selected_p50_at_most_30_seconds": true
    }

    execution-inputs.json

    {
      "source_sha": "c6bb3c5f6db3a31991f513239375a3edc8dd8b8e",
      "source_clean": true,
      "changed_source_validation_after_verified_view_window_fix": true,
      "sha256": {
        "/home/ubuntu/gf-beta-evidence/join-prepared-view-window-fixed/rebuild-copy-tool/src/main.rs": "179a2c6a9d663a6e64969e7f717e2ab0c1ad2095264dc14de6725dc5b395d288",
        "/home/ubuntu/gf-beta-evidence/join-prepared-view-window-fixed/rebuild-copy-tool/Cargo.toml": "00379a66fe591614964ecc7f7b96c487a9c9e6f2b8ff9d9138d509b65698ffb5",
        "/home/ubuntu/gf-beta-evidence/join-prepared-view-window-fixed/rebuild-copy-tool/Cargo.lock": "7c5b14ca224d8cb4e21a01f67ce1fe16cad708e9fc2a91c09a194f61a9a87e60",
        "/home/ubuntu/gf-beta-evidence/join-prepared-view-window-fixed/input-sha256-manifest.json": "23f8fdc0c4cb87bb4aadbc321702fc299e6131fae1ba40b598be6df549888092",
        "/home/ubuntu/gf-beta-scorecard-target/release/gf-beta-rebuild-copy": "631b445698c91ebf7a12924256e216b4390eb19bc4ded86f2c0371f80759fb56",
        "/home/ubuntu/gf-beta-scorecard-target/release/graphforge-benchmark-gdc-scorecard": "2cfcdaa6cccc2df5397507fc127d9cf16471c5050145e8fb8bfc671f500e59da",
        "/home/ubuntu/gf-beta-scorecard-target/release/gf": "d0a90f7269668b78f4567be313cb731f407a983060008cf095c8fea3486fd7ef",
        "/home/ubuntu/gf-beta-evidence/join-candidate-view-window-fixed/ic3-workload.json": "d0ad62c9ef96260bdb4ed224e0c5ba0b75ad9905cc12e7b6f764be1b4cd0ef57",
        "/home/ubuntu/gf-beta-evidence/join-candidate-view-window-fixed/ic3-reference.json": "723cde5975f11f290fc6ad342b8dfdb7172a3cc18b85de1200ebcc5cd3d7010f",
        "/home/ubuntu/gf-beta-evidence/join-prepared-view-window-fixed/snb-interactive/expected-counts.json": "9daa4440bfbac2f7d3b8165445f4afab67890cb71267d85a7691680269926feb"
      }
    }

    native-ic3/invocation.json

    {
      "source": "/home/ubuntu/gf-join-finish",
      "source_sha": "c6bb3c5f6db3a31991f513239375a3edc8dd8b8e",
      "command": [
        "/home/ubuntu/gf-beta-scorecard-target/release/graphforge-benchmark-gdc-scorecard",
        "query",
        "--project",
        "/home/ubuntu/gf-beta-evidence/join-candidate-view-window-fixed/ic",
        "--workload",
        "/home/ubuntu/gf-beta-evidence/join-candidate-view-window-fixed/ic3-workload.json",
        "--expected-counts",
        "/home/ubuntu/gf-beta-evidence/join-prepared-view-window-fixed/snb-interactive/expected-counts.json",
        "--output",
        "/home/ubuntu/gf-beta-evidence/join-candidate-view-window-fixed/ic3-evidence.json",
        "--results-dir",
        "/home/ubuntu/gf-beta-evidence/join-candidate-view-window-fixed/ic3-results",
        "--query-timeout-seconds",
        "120"
      ],
      "command_sha256": "2cfcdaa6cccc2df5397507fc127d9cf16471c5050145e8fb8bfc671f500e59da",
      "wrapper_sha256": "97e0cae501b3f49ae49f62a021c208c3555eedb7201f2181161817a30c3590a5",
      "phase_helper_sha256": "50240560bee83ead3208c62e6ffda0993014dad05495060e30453fbe4aa6a46b",
      "benchexec_version": "3.35",
      "python": "/usr/bin/python3",
      "python_sha256": "52e0a13e60a981d8c4b6478be2ba5176f69da07948a056bf49cf6f077e30cb41",
      "memlimit": 4294967296,
      "walltimelimit": 14400,
      "working_dir": "/home/ubuntu/gf-beta-evidence/join-candidate-view-window-fixed",
      "output": "/home/ubuntu/gf-beta-evidence/join-candidate-view-window-fixed/native-ic3",
      "writable_project": "/home/ubuntu/gf-beta-evidence/join-candidate-view-window-fixed",
      "container": true,
      "root_mode": "read-only",
      "container_tmpfs": false,
      "environment": {
        "PATH": "/usr/bin:/bin",
        "LANG": "C.UTF-8",
        "LC_ALL": "C.UTF-8",
        "TMPDIR": "/home/ubuntu/gf-beta-evidence/join-candidate-view-window-fixed/native-ic3/tmp"
      },
      "host_filesystem": "{\n   \"filesystems\": [\n      {\n         \"target\": \"/\",\n         \"source\": \"/dev/md3\",\n         \"fstype\": \"ext4\",\n         \"options\": \"rw,relatime\"\n      }\n   ]\n}\n"
    }

    native-ic3/runexecutor-result.json

    {
      "starttime": "2026-10-14 17:24:11.763720+00:00",
      "walltime": 19.253061415976845,
      "cputime": 59.779501,
      "memory": 2647412736,
      "blkio-read": 807706624,
      "blkio-write": 3290451968,
      "pressure-cpu-some": 0.098013,
      "pressure-memory-some": 0.0,
      "pressure-io-some": 0.174139,
      "exitcode": "return value 0",
      "exit_value": 0,
      "exit_signal": null
    }

    native-ic3/native-cgroup-facts.json

    {
      "runs": {
        "/sys/fs/cgroup/user.slice/user-1000.slice/[email protected]/benchexec.slice/run-p2052196-i39826230.scope/benchmark_4m_qzbdh": {
          "cgroup_path": "/sys/fs/cgroup/user.slice/user-1000.slice/[email protected]/benchexec.slice/run-p2052196-i39826230.scope/benchmark_4m_qzbdh",
          "memory.max": "4294967296",
          "memory.swap.max": "0",
          "cgroup.procs": "",
          "memory.events": "low 0\nhigh 0\nmax 0\noom 0\noom_kill 0\noom_group_kill 0\nsock_throttled 0",
          "observed_pids": []
        }
      },
      "observer_errors": [],
      "requested_memory_bytes": 4294967296,
      "observed_4GiB": true,
      "swap_observed": true,
      "swapped": false
    }

    ic3-workload.json

    {
      "schema": "graphforge-gdc-query-workload/1",
      "suite": "snb-interactive",
      "variants": [
        {
          "bindings": [
            {
              "id": "p000",
              "params": {
                "countryXName": {
                  "type": "Str",
                  "value": "Laos"
                },
                "countryYName": {
                  "type": "Str",
                  "value": "Scotland"
                },
                "endDate": {
                  "type": "Int",
                  "value": 1310515200000
                },
                "personId": {
                  "type": "Int",
                  "value": 17592186055119
                },
                "startDate": {
                  "type": "Int",
                  "value": 1306886400000
                }
              }
            },
            {
              "id": "p001",
              "params": {
                "countryXName": {
                  "type": "Str",
                  "value": "Uruguay"
                },
                "countryYName": {
                  "type": "Str",
                  "value": "Scotland"
                },
                "endDate": {
                  "type": "Int",
                  "value": 1301356800000
                },
                "personId": {
                  "type": "Int",
                  "value": 8796093030404
                },
                "startDate": {
                  "type": "Int",
                  "value": 1298937600000
                }
              }
            },
            {
              "id": "p002",
              "params": {
                "countryXName": {
                  "type": "Str",
                  "value": "Scotland"
                },
                "countryYName": {
                  "type": "Str",
                  "value": "Slovakia"
                },
                "endDate": {
                  "type": "Int",
                  "value": 1304208000000
                },
                "personId": {
                  "type": "Int",
                  "value": 2199023266354
                },
                "startDate": {
                  "type": "Int",
                  "value": 1301616000000
                }
              }
            }
          ],
          "id": "IC3",
          "operation": {
            "kind": "cypher",
            "text": "MATCH (countryX:Country {name: $countryXName}),\n      (countryY:Country {name: $countryYName}),\n      (person:Person {id: $personId})\nWITH person, countryX, countryY\nLIMIT 1\nMATCH (city:City)-[:IS_PART_OF]->(country:Country)\nWHERE country IN [countryX, countryY]\nWITH person, countryX, countryY, collect(city) AS cities\nMATCH (person)-[:KNOWS*1..2]-(friend)-[:IS_LOCATED_IN]->(city)\nWHERE NOT person = friend AND NOT city IN cities\nWITH DISTINCT friend, countryX, countryY\nMATCH (friend)<-[:HAS_CREATOR]-(message),\n      (message)-[:IS_LOCATED_IN]->(country)\nWHERE $endDate > message.creationDate >= $startDate AND\n      country IN [countryX, countryY]\nWITH friend,\n     CASE WHEN country = countryX THEN 1 ELSE 0 END AS messageX,\n     CASE WHEN country = countryY THEN 1 ELSE 0 END AS messageY\nWITH friend, sum(messageX) AS xCount, sum(messageY) AS yCount\nWHERE xCount > 0 AND yCount > 0\nRETURN friend.id AS friendId,\n       friend.firstName AS friendFirstName,\n       friend.lastName AS friendLastName,\n       xCount,\n       yCount,\n       xCount + yCount AS xyCount\nORDER BY xyCount DESC, friendId ASC\nLIMIT 20"
          },
          "ordered": true,
          "columns": [
            "friendId",
            "friendFirstName",
            "friendLastName",
            "xCount",
            "yCount",
            "xyCount"
          ]
        }
      ]
    }

    ic3-reference.json

    {
      "queries": {
        "IC3": {
          "bindings": {
            "p000": {
              "columns": [
                "friendId",
                "friendFirstName",
                "friendLastName",
                "xCount",
                "yCount",
                "xyCount"
              ],
              "rows": [
                [
                  "8796093029689",
                  "Eun-Hye",
                  "Yoon",
                  "1",
                  "1",
                  "2"
                ]
              ]
            },
            "p001": {
              "columns": [
                "friendId",
                "friendFirstName",
                "friendLastName",
                "xCount",
                "yCount",
                "xyCount"
              ],
              "rows": []
            },
            "p002": {
              "columns": [
                "friendId",
                "friendFirstName",
                "friendLastName",
                "xCount",
                "yCount",
                "xyCount"
              ],
              "rows": []
            }
          },
          "matching": "exact"
        }
      },
      "rung_id": "sf1",
      "schema": "graphforge-gdc-rung-reference/1",
      "source": "a spec-derived reference (gdc_snb_interactive_reference over the loaded snapshot), not the LDBC validation set (the pinned validation stream begins with an update at position 0)",
      "suite_id": "snb-interactive"
    }

    00000000.json

    {"binding_id":"p000","columns":[{"name":"friendId","type":"Int64"},{"name":"friendFirstName","type":"Utf8"},{"name":"friendLastName","type":"Utf8"},{"name":"xCount","type":"Int64"},{"name":"yCount","type":"Int64"},{"name":"xyCount","type":"Int64"}],"ordered":true,"query_id":"IC3","result_sha256":"39fe4a987576dc1620e3aaad2b700fbdadc77f5cc67e9523e38bbe251afadf69","rows":[["8796093029689","Eun-Hye","Yoon","1","1","2"]],"schema":"graphforge-gdc-query-result/1"}

    00000001.json

    {"binding_id":"p001","columns":[{"name":"friendId","type":"Int64"},{"name":"friendFirstName","type":"Utf8"},{"name":"friendLastName","type":"Utf8"},{"name":"xCount","type":"Int64"},{"name":"yCount","type":"Int64"},{"name":"xyCount","type":"Int64"}],"ordered":true,"query_id":"IC3","result_sha256":"0c50a11692c6f222c4fa3c454f546946bd4ac403cd1be936d74e18b2859cbfcd","rows":[],"schema":"graphforge-gdc-query-result/1"}

    00000002.json

    {"binding_id":"p002","columns":[{"name":"friendId","type":"Int64"},{"name":"friendFirstName","type":"Utf8"},{"name":"friendLastName","type":"Utf8"},{"name":"xCount","type":"Int64"},{"name":"yCount","type":"Int64"},{"name":"xyCount","type":"Int64"}],"ordered":true,"query_id":"IC3","result_sha256":"0c50a11692c6f222c4fa3c454f546946bd4ac403cd1be936d74e18b2859cbfcd","rows":[],"schema":"graphforge-gdc-query-result/1"}
  10. DecisionNerd commented on Oct 14, 2026

    @DecisionNerd
    ContributorAuthor

    The complete final C6 IC3 diagnostic bundle is now published: https://gist.github.com/DecisionNerd/461fa55cf5b1aef762584b7a4a45db01 . All 34 nonempty files were verified byte-for-byte after publication; the manifest records three omitted empty files. It includes native cgroup records, all three emitted results and references, exact input/binary hashes, the 59-test/check logs, completed CodeRabbit review, and the old-code regression reproduction.

    Coverage remains IC3’s first three bindings only (3/600 Interactive bindings), with all three reference matches and certification=false. This is the final reviewed candidate’s evidence; full-suite beta coverage remains outstanding on #1982. The bundle preserves the queue state observed at staging and makes no merge claim.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcoreCore source code changesexecutorChanges to query executor

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions