Skip to content

Commit ac04acc

Browse files
luisleo526claude
andcommitted
Equity curve window: pin the reset of the block before a block read, and two documentation fixes
The window clears its block before it reads a replacement and publishes the new one only after the read completes, so a read that fails after a short read cannot leave a half-overwritten block marked valid. No row exercised that: the existing failed-refill rows never reach an adjacent block fill that fails. The new row walks the first block through a one-line cache, shortens the backing file inside the next block, expects the typed I/O failure on the adjacent lookup, and then requires an intact early point to read back bit for bit (without the reset it would return the bytes of the partially overwritten block). The equity writer's citation in the arena bound page pointed at the line that is now another function; it names the writer's write line. The cache parameter page now says what the documented bypass costs: zero capacity turns the cache off, so each lookup reads one point. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
1 parent a1a2abe commit ac04acc

3 files changed

Lines changed: 67 additions & 2 deletions

File tree

‎docs/design/arena-bound.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -338,7 +338,7 @@ indexPeak(k) = 24 * slots(k) # k > 32
338338

339339
| Physical user | Logical peak bytes | Code evidence |
340340
| --- | --- | --- |
341-
| Equity writer / selected hold-back / SpillCurve | 24*E | `pf_equity_point_t` size assertion `include/pineforge/metrics_stream.hpp:16`; `src/report_scratch.cpp:232` |
341+
| Equity writer / selected hold-back / SpillCurve | 24*E | `pf_equity_point_t` size assertion `include/pineforge/metrics_stream.hpp:16`; `src/report_scratch.cpp:238` |
342342
| Position tail | (30+L)*T | `src/source/pine_position_tail.hpp:20`; footer packing at `src/source/pine_position_tail.cpp:94` |
343343
| Cohort AVL and five-word summary | (96+L)*D + 1[D>0] | `include/pineforge/detail/disk_id_table.hpp:109`; `CohortFacts::Record` at `include/pineforge/source/pine_adapter.hpp:2059-2062` |
344344
| Entry-rank AVL | (60+L)*J + 1[J>0] | `exit_rank_keys_` at `src/source/pine_adapter.cpp:6773-6774`; `include/pineforge/detail/disk_id_table.hpp:109` |

‎docs/pages/memory-management.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -508,7 +508,7 @@ This setter adds no environment variables or JSON limits.
508508
| Erased minima and consumed cycles | `erased_minima_pages`, `consumed_cycle_pages` | 32 KiB each, shared by that owner's data/index files |
509509
| Position-tail read window | `position_tail_pages` | 64 KiB in 16 complete 4 KiB pages, plus one shared metadata charge per page |
510510
| Successor-root, group-receipt and chain-segment read caches | `successor_root_pages`, `receipt_tail_pages`, `chain_segment_pages` | 64, 64 and 16 fixed-width payload rows respectively; pending write batches are separate fixed transaction workspace |
511-
| Ordinary report-sink metric curve reader | `equity_curve_pages` | 2,730 complete 24-byte point lines, plus a fixed 65,520-byte sequential block workspace charged by `cache_control_bytes()`; the writer's retry batch is separate mandatory workspace |
511+
| Ordinary report-sink metric curve reader | `equity_curve_pages` | 2,730 complete 24-byte point lines, plus a fixed 65,520-byte sequential block workspace charged by `cache_control_bytes()`; zero capacity turns the cache off, so each lookup reads one point (about eight backing reads per point of a report); the writer's retry batch is separate mandatory workspace |
512512
| Numeric Series/rings, HeldHistory, extrema and WMA windows constructed by the run | `ta_history_pages` | Existing owning logical window; finite allowances select backed numeric pages |
513513
| Native-day trade-day/week/month columns | `feed_calendar_pages` | 256 KiB inclusive per column |
514514
| Recorded request tapes installed without an explicit C++ capacity | `recorded_series_pages` | 256 KiB inclusive per tape |

‎tests/test_curve_window_sizes.cpp‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -305,6 +305,70 @@ void failed_refill(const char* directory, std::size_t bytes) {
305305
"failed refill preserves every unchanged early point");
306306
require(equal(curve[curve.size() - 1], points.back()), "repaired cold backing is retryable");
307307
}
308+
309+
void failed_adjacent_block_refill(const char* directory) {
310+
rs::ScratchFile backing(directory);
311+
const auto points = bit_pattern_points(2 * rs::kScratchWindowPoints + 1);
312+
backing.write(0, points.data(), points.size() * sizeof(points[0]));
313+
const pineforge::metrics::ScratchCurveRegion region{
314+
backing.descriptor(), 0, static_cast<std::int64_t>(points.size())};
315+
rs::ScratchReadStats inner;
316+
rs::ScratchCurveWindow curve(region, sizeof(pf_equity_point_t), &inner);
317+
// One point of payload reserves exactly one complete shared cache line.
318+
require(curve.cache_capacity_bytes() == sizeof(pf_equity_point_t),
319+
"adjacent short-read row uses a single point cache line");
320+
// The recovery probe must distinguish the old block from its replacement.
321+
require(!equal(points[5], points[rs::kScratchWindowPoints + 5]),
322+
"old and replacement block probes have distinct bits");
323+
// Lookup 0 reads and retains one point; lookup 1 is already an adjacent
324+
// miss and fills block 0. Every expected point was written above verbatim.
325+
for (std::size_t i = 0; i != rs::kScratchWindowPoints; ++i)
326+
require(equal(curve[static_cast<std::int64_t>(i)], points[i]),
327+
"prime the complete old block with exact point bytes");
328+
// One initial point plus one complete block; later misses use that block.
329+
require(inner.bytes == (rs::kScratchWindowPoints + 1) * sizeof(points[0]),
330+
"forward walk fills the aligned old block");
331+
332+
const auto cut = (rs::kScratchWindowPoints + 2000) * sizeof(points[0])
333+
+ sizeof(points[0]) / 2;
334+
// This open descriptor can be shortened inside block 1, after point 5
335+
// of that block and halfway through its point 2000.
336+
require(::ftruncate(backing.descriptor(), static_cast<off_t>(cut)) == 0,
337+
"truncate the adjacent block inside a point");
338+
const auto before = inner.calls;
339+
const auto before_bytes = inner.bytes;
340+
bool failed = false;
341+
try {
342+
// The preceding lookup was the last point of block 0, so this miss
343+
// requests all of block 1 rather than its intact first point alone.
344+
(void)curve[static_cast<std::int64_t>(rs::kScratchWindowPoints)];
345+
} catch (const std::system_error& error) {
346+
const auto failure = pineforge::classify_run_failure(error);
347+
// read_exact reaches EOF after a positive prefix and raises EIO
348+
// through ReportScratchIoError, whose code is report_sink_failed.
349+
require(error.code().value() == EIO
350+
&& failure.code == pineforge::RunFailureCode::report_sink_failed,
351+
"adjacent short block keeps its typed I/O failure");
352+
failed = true;
353+
}
354+
// EOF must throw after reaching the raw descriptor, not publish a block.
355+
require(failed && inner.calls > before,
356+
"short adjacent block read fails at the actual backing");
357+
// read_exact counts the 2000 complete points and half-point before EOF.
358+
require(inner.bytes - before_bytes
359+
== cut - rs::kScratchWindowPoints * sizeof(points[0]),
360+
"failed adjacent fill overwrites a positive block prefix");
361+
362+
const auto retry_bytes = inner.bytes;
363+
// Point 5's file bytes are intact and its shared line is cold. Clearing
364+
// count forces a reread; retaining block 0's tag would return point
365+
// kScratchWindowPoints + 5 from the overwritten array element instead.
366+
require(equal(curve[5], points[5]),
367+
"failed adjacent fill leaves no partially overwritten block in use");
368+
// Index 5 is nonadjacent to the last successful lookup, so only it is read.
369+
require(inner.bytes == retry_bytes + sizeof(points[0]),
370+
"recovery reads the intact early point from backing");
371+
}
308372
} // namespace
309373

310374
int main() {
@@ -337,6 +401,7 @@ int main() {
337401
for (const auto bytes : {std::size_t{0}, sizeof(pf_equity_point_t),
338402
2 * sizeof(pf_equity_point_t), rs::kScratchWindowBytes})
339403
failed_refill(directory.path, bytes);
404+
failed_adjacent_block_refill(directory.path);
340405
std::puts("curve capacity sweep passed");
341406
return 0;
342407
} catch (const std::exception& error) {

0 commit comments

Comments
 (0)