Skip to content

chore(core): assert all munmaps are succesful, add if-guards around non-compliant code#6092

Merged
bluestreak01 merged 59 commits intomasterfrom
mt_strict-munmap
Sep 5, 2025
Merged

chore(core): assert all munmaps are succesful, add if-guards around non-compliant code#6092
bluestreak01 merged 59 commits intomasterfrom
mt_strict-munmap

Conversation

@mtopolnik
Copy link
Copy Markdown
Contributor

@mtopolnik mtopolnik commented Sep 1, 2025

mmap

  • PR makes every call site checks the return value of mmap, and captures errno in CairoException.

  • On Windows, the mmap equivalent accepts len == 0 with the meaning "map the whole file". In order to maintain cross-platform consistency, make our mmap0 native function check len == 0 and return -1 instead, setting errno = ERROR_INVALID_PARAMETER.

  • PR adds an early check for len < 0, throwing CairoException if so.

mremap

Prior to this PR, mremap() was already checking for newSize < 0 and throwing CairoException. PR adds a proper test for this.

PR also pushes the check deeper down the call stack, into MmapCache, in order to ensure all calls to MmapCache.mremap() get the check.

munmap

Prior to this PR, munmap would return silently when called with address == 0, masking potential bugs. The PR makes munmap fail in that case, and ensures all call sites check address first.

General

  • Unifies the checking of the return value against MAP_FAILED. At various places there were == -1 and < 0 checks.
  • In a few places, puts munmap into finally to ensure cleanup
  • Cosmetic change: ex.errnoFileCannotRead -> ex.isFileCannotRead

Followups

  • add the same validation for ff.close(fd)
  • improve fuzz tests to inject failures for WAL table writing, as well as table reading. Currently it's only in non-WAL writing.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 1, 2025

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Add guards to avoid munmap on zero/invalid addresses and tighten mmap/mremap/unmap validation and error handling across Files, MmapCache, and multiple callers; introduce errno helpers and adjust related exception checks. No public API signatures were changed.

Changes

Cohort / File(s) Summary of changes
Mmap/unmap core and accounting
core/src/main/java/io/questdb/std/MmapCache.java, core/src/main/java/io/questdb/std/Files.java, core/src/main/java/io/questdb/std/FilesFacadeImpl.java
Stronger input validation for mmap/mremap/unmap (throw on invalid len/address), use FilesFacade.MAP_FAILED for failure checks, record memory accounting only on successful munmap, surface mmap failures as CairoException; rename/split errno helpers.
Files JNI / platform guards & tests
core/src/main/c/windows/files.c, core/src/test/java/io/questdb/test/FilesTest.java
Early guard for len==0 in Windows mmap0 native impl; new tests for invalid mmap/mremap/munmap and openCleanRW behaviors.
CairoException errno helpers
core/src/main/java/io/questdb/cairo/CairoException.java
Add ERRNO_INVALID_PARAMETER (POSIX/Windows), add isFileCannotRead() delegating to Files, remove errnoFileCannotRead() wrapper.
Callsites: guarded munmap & errno check updates
Multiple files:
core/src/main/java/io/questdb/griffin/engine/table/parquet/MappedMemoryPartitionDescriptor.java, core/src/main/java/io/questdb/cairo/O3PartitionJob.java, core/src/main/java/io/questdb/cairo/TableWriter.java, core/src/main/java/io/questdb/griffin/engine/functions/table/ReadParquetFunctionFactory.java, core/src/main/java/io/questdb/cutlass/text/CopyTask.java, core/src/main/java/io/questdb/cairo/wal/seq/TableTransactionLogV1.java, core/src/main/java/io/questdb/cairo/wal/seq/TableTransactionLogV2.java, core/src/main/java/io/questdb/cairo/wal/seq/TableTransactionLog.java, core/src/main/java/io/questdb/cairo/wal/WalWriter.java, core/src/main/java/io/questdb/cairo/wal/CheckWalTransactionsJob.java, core/src/main/java/io/questdb/cairo/TableUtils.java, core/src/main/java/io/questdb/cairo/TableNameRegistryStore.java, core/src/main/java/io/questdb/cairo/wal/WalPurgeJob.java, core/src/main/java/io/questdb/cairo/wal/seq/TableSequencerImpl.java, core/src/main/java/io/questdb/cairo/wal/seq/TableSequencerAPI.java, core/src/main/java/io/questdb/cairo/wal/seq/TableTransactionLogV2.java, core/src/main/java/io/questdb/griffin/engine/functions/catalogue/*.java, core/src/main/java/io/questdb/cairo/wal/CopyWalSegmentUtils.java
Replace unconditional munmap calls with guards checking address > 0 / != 0; change various CairoException errnoFileCannotRead() checks to isFileCannotRead(); ensure mmap failure is detected via MAP_FAILED and throw on failure; add try/finally to guarantee munmap on error in several paths; replace wildcard imports and tighten imports in one file.
Small C/C++ and build workflow tweaks
core/src/main/c/share/dedup.cpp, .github/workflows/rebuild_native_libs.yml
Minor control-flow: explicit return in lambda when merge_index==nullptr; macOS workflow step now uninstalls cmake before installing toolchains (commented as temporary fix).

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Caller as Caller (various)
  participant Mmap as Files / FilesFacade
  participant Cache as MmapCache
  Note over Caller,Mmap: Mapping path (mmap/mremap)
  Caller->>Mmap: mmap(... len)
  alt len invalid (<=0)
    Mmap-->>Caller: throw CairoException (invalid len)
  else success
    Mmap-->>Caller: addr (or MAP_FAILED -> throw)
  end

  Note over Caller,Cache: Unmap path
  Caller->>Cache: unmap(addr, len, tag)
  alt addr == 0 or len <= 0
    Cache-->>Caller: throw CairoException (invalid address/length)
  else
    Cache->>Cache: rc = munmap(addr,len)
    alt rc == success
      Cache->>Cache: recordMemAlloc(-len, tag)
      Cache-->>Caller: return
    else rc == failure
      Cache-->>Caller: throw CairoException(errno)
    end
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested labels

Core, storage

Suggested reviewers

  • jerrinot
  • bluestreak01
✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch mt_strict-munmap

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@mtopolnik
Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 1, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
core/src/main/java/io/questdb/std/MmapCache.java (1)

287-296: Asserting munmap failures is good; add a release-path signal and a len>0 assert.

The early return on address==0 and conditional accounting are solid. With assertions disabled in production, a failed munmap silently does nothing and leaks accounting. Consider emitting a log/warn (or metrics counter) on failure, and assert len>0 to catch misuse earlier.

Example minimal tweak:

 private static void unmap0(long address, long len, int memoryTag) {
     if (address == 0) {
         return;
     }
+    assert len > 0 : "munmap length must be > 0 [len=" + len + ']';
     int result = Files.munmap0(address, len);
     if (result != -1) {
         Unsafe.recordMemAlloc(-len, memoryTag);
     }
     assert result != -1 :
             "failed to munmap non-zero address [address=" + address + ", len=" + len + ", memoryTag=" + memoryTag + ']';
+    // Optional: emit a release-path signal when assertions are off.
+    // e.g., LOG.warn().$("munmap failed [address=").$(address).$(", len=").$(len).$(", tag=").$(memoryTag).I$();
 }
core/src/main/java/io/questdb/griffin/engine/table/parquet/MappedMemoryPartitionDescriptor.java (2)

55-59: Use overflow-safe sizing and add a sanity check before subtracting HEADER_SIZE.

Casting symbolOffsetsSize to int can overflow silently. Also, subtracting HEADER_SIZE assumes symbolOffsetsAddr ≥ HEADER_SIZE.

-            if (symbolOffsetsAddr != 0) {
-                final long symbolOffsetsSize = columnData.get(rawIndex + SYMBOL_OFFSET_SIZE_OFFSET);
-                final long offsetsMemSize = SymbolMapWriter.keyToOffset((int) symbolOffsetsSize + 1);
-                ff.munmap(symbolOffsetsAddr - SymbolMapWriter.HEADER_SIZE, offsetsMemSize, MemoryTag.MMAP_PARQUET_PARTITION_CONVERTER);
-            }
+            if (symbolOffsetsAddr != 0) {
+                final long symbolOffsetsSize = columnData.get(rawIndex + SYMBOL_OFFSET_SIZE_OFFSET);
+                // Ensure address math is valid and sizing is safe.
+                assert symbolOffsetsAddr >= SymbolMapWriter.HEADER_SIZE;
+                final int keyCapacity = Math.toIntExact(symbolOffsetsSize + 1);
+                final long offsetsMemSize = SymbolMapWriter.keyToOffset(keyCapacity);
+                ff.munmap(symbolOffsetsAddr - SymbolMapWriter.HEADER_SIZE, offsetsMemSize, MemoryTag.MMAP_PARQUET_PARTITION_CONVERTER);
+            }

55-59: Optional: reduce repetition of MemoryTag.

Capture MemoryTag.MMAP_PARQUET_PARTITION_CONVERTER in a local final int tag to make the loop a touch cleaner.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 9eb60b0 and 47d701d.

📒 Files selected for processing (2)
  • core/src/main/java/io/questdb/griffin/engine/table/parquet/MappedMemoryPartitionDescriptor.java (1 hunks)
  • core/src/main/java/io/questdb/std/MmapCache.java (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (26)
  • GitHub Check: New pull request (SelfHosted Other tests on linux-arm64)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests on linux-x64-zfs)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests on linux-arm64)
  • GitHub Check: New pull request (SelfHosted Other tests on linux-x64-zfs)
  • GitHub Check: New pull request (Hosted Running tests on windows-other)
  • GitHub Check: New pull request (Hosted Running tests on windows-pgwire)
  • GitHub Check: New pull request (Hosted Running tests on windows-cairo)
  • GitHub Check: New pull request (Hosted Running tests on windows-fuzz2)
  • GitHub Check: New pull request (Hosted Running tests on windows-fuzz1)
  • GitHub Check: New pull request (Hosted Running tests on windows-griffin)
  • GitHub Check: New pull request (Hosted Running tests on mac-other)
  • GitHub Check: New pull request (Hosted Running tests on mac-pgwire)
  • GitHub Check: New pull request (Hosted Running tests on mac-cairo-fuzz)
  • GitHub Check: New pull request (Hosted Running tests on mac-cairo)
  • GitHub Check: New pull request (SelfHosted Other tests Start ARM Agent)
  • GitHub Check: New pull request (SelfHosted Other tests Start X64Zfs Agent)
  • GitHub Check: New pull request (Rust Test and Lint on linux-jdk17)
  • GitHub Check: New pull request (Hosted Running tests on mac-griffin)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-other)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-pgwire)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-cairo)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-fuzz)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests Start X64Zfs Agent)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-griffin)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests Start ARM Agent)
  • GitHub Check: New pull request (Check Changes Check changes)
🔇 Additional comments (1)
core/src/main/java/io/questdb/griffin/engine/table/parquet/MappedMemoryPartitionDescriptor.java (1)

50-53: Good zero-addr guard; verify if primary column can ever be zero.

This prevents munmap(0) for columnSecondaryAddr. If there are scenarios where columnAddr (Line 45) can be 0, mirror the same guard there to avoid tripping the new assertion path.

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mtopolnik
❌ GitHub Actions - Rebuild Native Libraries


GitHub Actions - Rebuild Native Libraries seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

jerrinot
jerrinot previously approved these changes Sep 3, 2025
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 4, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (8)
core/src/main/java/io/questdb/std/MmapCache.java (3)

79-82: Unify failure checks to MAP_FAILED and fix misleading comment.

Current check uses -1 and comment says “return 0”. Use FilesFacade.MAP_FAILED and correct the comment.

-            if (address == -1) {
-                // mmap failed, return 0
-                return address;
-            }
+            if (address == FilesFacade.MAP_FAILED) {
+                // mmap failed
+                return address;
+            }

185-191: Replace -1 comparisons with FilesFacade.MAP_FAILED in mremap path.

Aligns with PR objective to unify MAP_FAILED checks.

-                    if (newAddress != -1) {
+                    if (newAddress != FilesFacade.MAP_FAILED) {
                         record.address = newAddress;
                         record.length = newSize;
                         record.memoryTag = memoryTag;
                         mmapAddrCache.removeAt(addrMapIndex);
                         mmapAddrCache.put(newAddress, record);
                     }
...
-                    if (newAddress != -1) {
+                    if (newAddress != FilesFacade.MAP_FAILED) {
                         record.count--;
                         MmapCacheRecord newRecord = createMmapCacheRecord(fd, mmapCacheKey, newSize, newAddress, memoryTag);
                         if (fdIndex != Integer.MAX_VALUE) {
                             mmapFileCache.putAt(fdIndex, mmapCacheKey, newRecord);
                         } else {
                             mmapFileCache.put(mmapCacheKey, newRecord);
                         }
                         mmapAddrCache.put(newAddress, newRecord);
                     }

Also applies to: 198-214


50-91: Replace raw -1 checks with FilesFacade.MAP_FAILED and switch invalid‐parameter throws to errnoInvalidParameter

  • core/src/main/java/io/questdb/std/MmapCache.java still uses raw if (address == -1)/if (newAddress != -1) in cacheMmap, mmap0, mremap0, and unmap0; replace these with comparisons against FilesFacade.MAP_FAILED (or the agreed facade constant) and propagate errors via the errno mechanism.
  • The initial len/offset validation in cacheMmap throws CairoException.critical(0)…put("invalid len"); update this to use Files.errnoInvalidParameter(…) instead.
core/src/main/java/io/questdb/griffin/engine/functions/catalogue/WalTransactionsFunctionFactory.java (1)

116-131: Prevent busy-spin: rethrow non-file-read CairoExceptions

Inside the retry loop, exceptions other than “file cannot read” are swallowed, causing an unbounded tight retry. Rethrow to avoid CPU spin and to surface genuine faults.

                 } catch (CairoException e) {
                     Misc.free(cursor);
                     if (e.isFileCannotRead()) {
                         // Txn sequencer can have its parts deleted due to housekeeping
                         // Need to keep scanning until we find a valid part
                         if (txnLo == 0) {
                             long writerTxn = executionContext.getCairoEngine().getTableSequencerAPI().getTxnTracker(tableToken).getWriterTxn();
                             if (writerTxn > 0) {
                                 txnLo = writerTxn;
                                 continue;
                             }
                         }
                         throw e;
                     }
+                    // Not a transient "file cannot read" condition — propagate.
+                    throw e;
                 }
core/src/main/java/io/questdb/cairo/TableWriter.java (3)

1719-1724: Add the same unmap guard here for consistency and safety.

convertPartitionParquetToNative() unmaps parquetAddr without validating it. Align this with other sites to avoid failing on addr == 0.

-            ff.munmap(parquetAddr, parquetSize, MemoryTag.MMAP_PARQUET_PARTITION_DECODER);
+            if (parquetAddr != 0) {
+                ff.munmap(parquetAddr, parquetSize, MemoryTag.MMAP_PARQUET_PARTITION_DECODER);
+            }

3479-3483: Guard this munmap as well.

If mapRO returns 0 (or throws before assignment is reliable), unmapping 0 will now fail. Guard to match PR’s policy.

-                    ff.munmap(mappedAddr, expectedFileSize, MemoryTag.MMAP_DEFAULT);
+                    if (mappedAddr != 0) {
+                        ff.munmap(mappedAddr, expectedFileSize, MemoryTag.MMAP_DEFAULT);
+                    }

3543-3548: Guard munmap in symbol column check.

Same rationale: avoid unmapping address 0.

-                ff.munmap(address, fileSize, MemoryTag.MMAP_DEFAULT);
+                if (address != 0) {
+                    ff.munmap(address, fileSize, MemoryTag.MMAP_DEFAULT);
+                }
core/src/main/java/io/questdb/cairo/wal/CopyWalSegmentUtils.java (1)

264-285: Ensure newAuxMemAddr is unmapped on all paths

If an exception is thrown between mapping and explicit unmap, the mapping can leak. Wrap in try/finally.

-            final long newAuxMemSize = columnTypeDriver.getAuxVectorSize(rowCount);
-            final long newAuxMemAddr = TableUtils.mapRW(ff, secondaryFd, newAuxMemSize, MEMORY_TAG);
-            ff.madvise(newAuxMemAddr, newAuxMemSize, Files.POSIX_MADV_RANDOM);
-
-            columnTypeDriver.shiftCopyAuxVector(
-                    dataStartOffset,
-                    auxMemAddr,
-                    startRowNumber,
-                    startRowNumber + rowCount - 1, // inclusive
-                    newAuxMemAddr,
-                    newAuxMemSize
-            );
-
-            columnRollSink.setSrcOffsets(dataStartOffset, columnTypeDriver.getAuxVectorSize(startRowNumber));
-            columnRollSink.setDestSizes(dataSize, newAuxMemSize);
-
-            if (commitMode != CommitMode.NOSYNC) {
-                ff.msync(newAuxMemAddr, newAuxMemSize, commitMode == CommitMode.ASYNC);
-            }
-            // All in memory calls, no need to unmap in finally
-            ff.munmap(newAuxMemAddr, newAuxMemSize, MEMORY_TAG);
-            return true;
+            final long newAuxMemSize = columnTypeDriver.getAuxVectorSize(rowCount);
+            final long newAuxMemAddr = TableUtils.mapRW(ff, secondaryFd, newAuxMemSize, MEMORY_TAG);
+            try {
+                ff.madvise(newAuxMemAddr, newAuxMemSize, Files.POSIX_MADV_RANDOM);
+
+                columnTypeDriver.shiftCopyAuxVector(
+                        dataStartOffset,
+                        auxMemAddr,
+                        startRowNumber,
+                        startRowNumber + rowCount - 1, // inclusive
+                        newAuxMemAddr,
+                        newAuxMemSize
+                );
+
+                columnRollSink.setSrcOffsets(dataStartOffset, columnTypeDriver.getAuxVectorSize(startRowNumber));
+                columnRollSink.setDestSizes(dataSize, newAuxMemSize);
+
+                if (commitMode != CommitMode.NOSYNC) {
+                    ff.msync(newAuxMemAddr, newAuxMemSize, commitMode == CommitMode.ASYNC);
+                }
+                return true;
+            } finally {
+                if (newAuxMemAddr > 0) {
+                    ff.munmap(newAuxMemAddr, newAuxMemSize, MEMORY_TAG);
+                }
+            }
🧹 Nitpick comments (29)
core/src/main/c/share/dedup.cpp (1)

635-643: Optional: express the null-scan with std::any_of for clarity.

Same behavior, slightly tighter and idiomatic.

-                [&](int data_index, int64_t lo, int64_t hi) -> bool {
-                    auto data = data_ptrs[data_index];
-                    for (int64_t i = lo; i < hi; i++) {
-                        if (data[i] != null_value) {
-                            return true;
-                        }
-                    }
-                    return false;
-                }
+                [&](int data_index, int64_t lo, int64_t hi) -> bool {
+                    const T* data = data_ptrs[data_index];
+                    return std::any_of(data + lo, data + hi,
+                                       [&](const T &v){ return v != null_value; });
+                }
core/src/main/java/io/questdb/cutlass/text/CopyTask.java (1)

1342-1344: Defensive reset after unmap

Set the local to 0 after unmapping to prevent accidental reuse.

-                if (mergeIndexAddr != 0) {
-                    ff.munmap(mergeIndexAddr, mergedIndexSize, MemoryTag.MMAP_IMPORT);
-                }
+                if (mergeIndexAddr != 0) {
+                    ff.munmap(mergeIndexAddr, mergedIndexSize, MemoryTag.MMAP_IMPORT);
+                    mergeIndexAddr = 0; // defensive
+                }
core/src/main/java/io/questdb/griffin/engine/functions/table/ReadParquetFunctionFactory.java (1)

100-102: Minor consistency/defense

Reset addr after unmap.

-                if (addr != 0) {
-                    ff.munmap(addr, fileSize, MemoryTag.MMAP_PARQUET_PARTITION_DECODER);
-                }
+                if (addr != 0) {
+                    ff.munmap(addr, fileSize, MemoryTag.MMAP_PARQUET_PARTITION_DECODER);
+                    addr = 0;
+                }
.github/workflows/rebuild_native_libs.yml (1)

33-35: Make cmake uninstall resilient

Avoid failures if cmake is absent or has dependencies.

-          # https://github.com/actions/runner-images/issues/12912
-          # Temporary fix (mtopolnik 2025-09-02): GH Action pre-installs a custom cmake version, install then fails
-          brew uninstall cmake
+          # https://github.com/actions/runner-images/issues/12912
+          # Temporary fix (mtopolnik 2025-09-02): GH Action pre-installs a custom cmake version, install then fails
+          if brew list --versions cmake >/dev/null 2>&1; then
+            brew uninstall --ignore-dependencies cmake || true
+          fi
core/src/main/c/windows/files.c (1)

703-711: Also guard negative len/offset (paranoia, matches Java checks)

Native layer can mirror Java validations for robustness.

     if (len == 0) {
         // With len == 0, Windows will mmap the whole file. To be POSIX-compatible, return MAP_FAILED instead.
         // docs: https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-mapviewoffileex
         // dwNumberOfBytesToMap: "If this parameter is 0 (zero), the mapping extends from the
         // specified offset to the end of the file mapping."
         SetLastError(ERROR_INVALID_PARAMETER);
         SaveLastError();
         return -1;
     }
+    if (len < 0 || offset < 0) {
+        SetLastError(ERROR_INVALID_PARAMETER);
+        SaveLastError();
+        return -1;
+    }
core/src/main/java/io/questdb/cairo/O3PartitionJob.java (2)

267-269: Reset address after unmap

Small defensive cleanup.

-                if (parquetAddr != 0) {
-                    ff.munmap(parquetAddr, parquetSize, MemoryTag.MMAP_PARQUET_PARTITION_DECODER);
-                }
+                if (parquetAddr != 0) {
+                    ff.munmap(parquetAddr, parquetSize, MemoryTag.MMAP_PARQUET_PARTITION_DECODER);
+                    parquetAddr = 0;
+                }

2570-2572: Nit: reset address; plus consider zero-size fast-path

  • Reset local after unmap.
  • Optionally skip mapping when newParquetSize == 0 to avoid needless exceptions.
-            if (parquetAddr != 0) {
-                ff.munmap(parquetAddr, newParquetSize, MemoryTag.MMAP_PARQUET_PARTITION_DECODER);
-            }
+            if (parquetAddr != 0) {
+                ff.munmap(parquetAddr, newParquetSize, MemoryTag.MMAP_PARQUET_PARTITION_DECODER);
+                parquetAddr = 0;
+            }

Additional (outside this hunk) suggestion:

// early in updateParquetIndexes, before mapping
if (newParquetSize == 0) {
    return;
}
core/src/main/java/io/questdb/std/MmapCache.java (8)

51-58: Prefer errnoInvalidParameter over errno=0 for invalid len.

Use the dedicated errno to improve diagnostics and align with cross-platform consistency goals.

-        if (len < 1) {
-            throw CairoException.critical(0)
+        if (len < 1) {
+            throw CairoException.critical(Files.errnoInvalidParameter())
                     .put("could not mmap file, invalid len [len=").put(len)
                     .put(", offset=").put(offset)
                     .put(", fd=").put(fd)
                     .put(", memoryTag=").put(memoryTag)
                     .put(']');
         }

112-121: Use errnoInvalidParameter for invalid newSize.

Same rationale as for len above.

-        if (newSize < 1) {
-            throw CairoException.critical(0)
+        if (newSize < 1) {
+            throw CairoException.critical(Files.errnoInvalidParameter())
                     .put("could not remap file, invalid newSize [previousSize=").put(previousSize)
                     .put(", newSize=").put(newSize)
                     .put(", offset=").put(offset)
                     .put(", fd=").put(fd)
                     .put(", memoryTag=").put(memoryTag)
                     .put(']');
         }

231-235: Return EINVAL-like errno for invalid unmap arguments.

Use the new errno helper instead of 0.

-        if (address <= 0 || len <= 0) {
-            throw CairoException.critical(0)
+        if (address <= 0 || len <= 0) {
+            throw CairoException.critical(Files.errnoInvalidParameter())
                     .put("unmap: invalid address or length [address=" + address + ", len=" + len + ']');
         }

293-304: Unify MAP_FAILED check in mremap accounting.

Mirror mmap path by using FilesFacade.MAP_FAILED.

-        if (address != -1) {
+        if (address != FilesFacade.MAP_FAILED) {
             if (oldMemoryTag == memoryTag) {
                 Unsafe.recordMemAlloc(newSize - previousSize, memoryTag);
             } else {
                 Unsafe.recordMemAlloc(newSize, memoryTag);
                 Unsafe.recordMemAlloc(-previousSize, oldMemoryTag);
             }
         }

306-316: Treat munmap success explicitly (result == 0); throw on non‑zero.

This is clearer and avoids accidental success if return contract ever changes.

-        int result = Files.munmap0(address, len);
-        if (result != -1) {
+        int result = Files.munmap0(address, len);
+        if (result == 0) {
             Unsafe.recordMemAlloc(-len, memoryTag);
         } else {
             throw CairoException.critical(Os.errno())
                     .put("munmap failed [address=").put(address)
                     .put(", len=").put(len)
                     .put(", memoryTag=").put(memoryTag).put(']');
         }

281-283: Update comment: unmap is executed outside the synchronized section, not “offloaded”.

The current comment suggests a background worker that doesn’t exist here.

-        // offload the unmap to a single thread to not block everyone under synchronized section
+        // perform unmap outside the synchronized section to avoid blocking other threads
         unmap0(unmapPtr, unmapLen, unmapTag);

50-61: Optional: validate offset >= 0 for early parameter sanity.

Negative offsets will fail in native anyway; failing early yields clearer errors and avoids syscalls.


122-124: Potential accounting mismatch when bypassing cache.

When bypassing cache, mremap0(..., oldMemoryTag=memoryTag, memoryTag) assumes unchanged tag. If callers ever change tags across remaps, memory accounting will be off.

Would you like a follow-up patch to plumb the previous tag (or document the invariant that tags must not change across non-cached remaps)?

core/src/main/java/io/questdb/cairo/wal/CheckWalTransactionsJob.java (1)

106-109: Optional: log swallowed race for observability

You correctly gate on isFileCannotRead(). Consider a low-level log to aid production triage when this race is frequent.

-                    } catch (CairoException e) {
-                        if (!e.isFileCannotRead()) {
-                            throw e;
-                        } // race, table is dropped, ApplyWal2TableJob is already deleting the files
-                    }
+                    } catch (CairoException e) {
+                        if (!e.isFileCannotRead()) {
+                            throw e;
+                        }
+                        // race, table is dropped, ApplyWal2TableJob is already deleting the files
+                        if (LOG.isEnabled(LogLevel.DEBUG)) {
+                            LOG.debug().$("txn file unreadable during check (likely drop in progress) [table=")
+                                    .$(tableToken).I$();
+                        }
+                    }
core/src/main/java/io/questdb/cairo/wal/WalPurgeJob.java (1)

427-433: Minor: capture errno once to keep logs consistent

Avoid calling ff.errno() twice; capture once after the failing call so the message reflects the same errno used in the condition.

-        if (!ff.rmdir(path, false) && !Files.isErrnoFileDoesNotExist(ff.errno())) {
-            LOG.debug()
-                    .$("could not delete directory [path=").$(path)
-                    .$(", errno=").$(ff.errno())
-                    .I$();
+        final boolean removed = ff.rmdir(path, false);
+        final int errno = removed ? 0 : ff.errno();
+        if (!removed && !Files.isErrnoFileDoesNotExist(errno)) {
+            LOG.debug()
+                    .$("could not delete directory [path=").$(path)
+                    .$(", errno=").$(errno)
+                    .I$();
             return false;
         }
core/src/main/java/io/questdb/cairo/wal/seq/TableTransactionLogV1.java (2)

372-379: Use ff.errno() for consistency with FilesFacade-backed calls

For map failures, prefer ff.errno() (already used elsewhere) to keep errno sourcing consistent with the facade.

-                if (newAddr == FilesFacade.MAP_FAILED) {
-                    throw CairoException.critical(Os.errno()).put("cannot mmap transaction log [path=").put(path).put(']');
+                if (newAddr == FilesFacade.MAP_FAILED) {
+                    throw CairoException.critical(ff.errno()).put("cannot mmap transaction log [path=").put(path).put(']');
                 }

391-395: Same here: prefer ff.errno() on remap failure

-            if (newAddr == FilesFacade.MAP_FAILED) {
-                throw CairoException.critical(Os.errno()).put("cannot remap transaction log [fd=").put(fd).put(']');
+            if (newAddr == FilesFacade.MAP_FAILED) {
+                throw CairoException.critical(ff.errno()).put("cannot remap transaction log [fd=").put(fd).put(']');
             }
core/src/main/java/io/questdb/cairo/wal/WalWriter.java (1)

1759-1766: Minor nit: avoid repeating the async check

Cache the async flag once to reduce duplication.

-                if (commitMode != CommitMode.NOSYNC) {
-                    ff.msync(auxMemAddr, auxMemSize, commitMode == CommitMode.ASYNC);
-                }
+                final boolean async = commitMode == CommitMode.ASYNC;
+                if (commitMode != CommitMode.NOSYNC) {
+                    ff.msync(auxMemAddr, auxMemSize, async);
+                }
...
-                if (commitMode != CommitMode.NOSYNC) {
-                    ff.msync(dataMemAddr, varColSize, commitMode == CommitMode.ASYNC);
-                }
+                final boolean async = commitMode == CommitMode.ASYNC;
+                if (commitMode != CommitMode.NOSYNC) {
+                    ff.msync(dataMemAddr, varColSize, async);
+                }

Also applies to: 1776-1783

core/src/main/java/io/questdb/cairo/TableWriter.java (1)

144-146: Remove redundant static imports or the wildcard to avoid duplication.

You have both import static io.questdb.cairo.TableUtils.*; and explicit openAppend/openRO static imports. Prefer one style; given the wildcard, the two explicit imports are redundant.

 import static io.questdb.cairo.TableUtils.*;
-import static io.questdb.cairo.TableUtils.openAppend;
-import static io.questdb.cairo.TableUtils.openRO;
core/src/main/java/io/questdb/cairo/wal/seq/TableTransactionLog.java (1)

343-360: Unify errno source and enrich context in mmap failure path

Prefer ff.errno() (used elsewhere in this PR) over Os.errno() and include txnMetaOffset in the error to aid diagnosis.

-                            } else {
-                                close();
-                                throw CairoException.critical(Os.errno())
-                                        .put("cannot mmap table transaction log [path=").put(path)
-                                        .put(", txnMetaOffsetHi=").put(txnMetaOffsetHi)
-                                        .put(']');
+                            } else {
+                                close();
+                                throw CairoException.critical(ff.errno())
+                                        .put("cannot mmap table transaction log [path=").put(path)
+                                        .put(", txnMetaOffsetLo=").put(txnMetaOffset)
+                                        .put(", txnMetaOffsetHi=").put(txnMetaOffsetHi)
+                                        .put(']');
core/src/main/java/io/questdb/cairo/wal/CopyWalSegmentUtils.java (2)

158-163: Guard fsync on optional FDs

srcVarFd/dstVarFd can be -1 for fixed-size targets. Guard to avoid unnecessary syscalls/errors.

-                if (commitMode != CommitMode.NOSYNC) {
-                    ff.fsync(srcFixFd);
-                    ff.fsync(srcVarFd);
-                    ff.fsync(dstFixFd);
-                    ff.fsync(dstVarFd);
-                }
+                if (commitMode != CommitMode.NOSYNC) {
+                    ff.fsync(srcFixFd);
+                    if (srcVarFd > -1) ff.fsync(srcVarFd);
+                    ff.fsync(dstFixFd);
+                    if (dstVarFd > -1) ff.fsync(dstVarFd);
+                }

224-231: Defensive unmap check

Given munmap now fails on address == 0, add a guard to avoid spurious failures if the returned address ever equals 0.

-        try {
+        try {
             Vect.flattenIndex(srcDataTimestampAddr, rowCount);
             if (commitMode != CommitMode.NOSYNC) {
                 ff.msync(srcDataTimestampAddr, size, commitMode == CommitMode.ASYNC);
             }
         } finally {
-            ff.munmap(srcDataTimestampAddr, size, MEMORY_TAG);
+            if (srcDataTimestampAddr > 0) {
+                ff.munmap(srcDataTimestampAddr, size, MEMORY_TAG);
+            }
         }
core/src/main/java/io/questdb/cairo/TableUtils.java (1)

1108-1110: Add runtime guard before munmap

An assert may be disabled in production. Since munmap(0, ...) now fails, add a guard to avoid false-positive failures.

-        long adjustedAddress = address - alignedExtraLen;
-        assert adjustedAddress > 0 : "address <= alignedExtraLen";
-        ff.munmap(adjustedAddress, size + alignedExtraLen, memoryTag);
+        long adjustedAddress = address - alignedExtraLen;
+        assert adjustedAddress > 0 : "address <= alignedExtraLen";
+        if (adjustedAddress > 0) {
+            ff.munmap(adjustedAddress, size + alignedExtraLen, memoryTag);
+        }
core/src/main/java/io/questdb/std/Files.java (1)

287-295: Include POSIX EACCES in isErrnoFileCannotRead()
No existing POSIX “permission denied” constant—define, for example,

// in CairoException
public static final int ERRNO_ACCESS_DENIED_POSIX = 13;

then update Files.isErrnoFileCannotRead() to also check

|| errno == CairoException.ERRNO_ACCESS_DENIED_POSIX
core/src/test/java/io/questdb/test/FilesTest.java (3)

768-793: Avoid masking the root failure by calling munmap with len=0 in the success path.

If mmap erroneously succeeds, the subsequent munmap(..., 0, ...) throws a different error and can obscure the diagnosis. Let Assert.fail fire immediately after mmap instead.

-                    long mmapAddr = Files.mmap(fdro, 0, 0, Files.MAP_RO, MemoryTag.MMAP_DEFAULT);
-                    Files.munmap(mmapAddr, 0, MemoryTag.MMAP_DEFAULT);
+                    long mmapAddr = Files.mmap(fdro, 0, 0, Files.MAP_RO, MemoryTag.MMAP_DEFAULT);
                     Assert.fail("mmap with zero len should have failed");

795-827: Don’t overwrite the original mapping and drop the zero-len munmap.

Overwriting mmapAddr complicates cleanup; the zero-length munmap similarly hides the intended failure. Keep the original address for cleanup and fail immediately if mremap doesn’t throw.

-                        mmapAddr = Files.mremap(fdro, mmapAddr, 64, 0, 0, Files.MAP_RO, MemoryTag.MMAP_DEFAULT);
-                        Files.munmap(mmapAddr, 0, MemoryTag.MMAP_DEFAULT);
+                        long ignored = Files.mremap(fdro, mmapAddr, 64, 0, 0, Files.MAP_RO, MemoryTag.MMAP_DEFAULT);
                         Assert.fail("mremap with len 0 should have failed");

869-885: Assert FDs are valid after openCleanRW.

Minor robustness: assert the returned FDs are positive before using/closing.

-                long fd = Files.openCleanRW(path.$(), 1024);
+                long fd = Files.openCleanRW(path.$(), 1024);
+                Assert.assertTrue(fd > 0);
                 Assert.assertTrue(Files.exists(path.$()));
                 Assert.assertEquals(1024, Files.length(path.$()));
 
-                long fd2 = Files.openCleanRW(path.$(), 2048);
+                long fd2 = Files.openCleanRW(path.$(), 2048);
+                Assert.assertTrue(fd2 > 0);
                 Assert.assertEquals(2048, Files.length(path.$()));
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 47d701d and d70e452.

⛔ Files ignored due to path filters (5)
  • core/src/main/resources/io/questdb/bin/darwin-aarch64/libquestdb.dylib is excluded by !**/*.dylib
  • core/src/main/resources/io/questdb/bin/darwin-x86-64/libquestdb.dylib is excluded by !**/*.dylib
  • core/src/main/resources/io/questdb/bin/linux-aarch64/libquestdb.so is excluded by !**/*.so
  • core/src/main/resources/io/questdb/bin/linux-x86-64/libquestdb.so is excluded by !**/*.so
  • core/src/main/resources/io/questdb/bin/windows-x86-64/libquestdb.dll is excluded by !**/*.dll
📒 Files selected for processing (26)
  • .github/workflows/rebuild_native_libs.yml (1 hunks)
  • core/src/main/c/share/dedup.cpp (1 hunks)
  • core/src/main/c/windows/files.c (1 hunks)
  • core/src/main/java/io/questdb/cairo/CairoException.java (2 hunks)
  • core/src/main/java/io/questdb/cairo/O3PartitionJob.java (2 hunks)
  • core/src/main/java/io/questdb/cairo/TableNameRegistryStore.java (2 hunks)
  • core/src/main/java/io/questdb/cairo/TableUtils.java (5 hunks)
  • core/src/main/java/io/questdb/cairo/TableWriter.java (5 hunks)
  • core/src/main/java/io/questdb/cairo/wal/CheckWalTransactionsJob.java (1 hunks)
  • core/src/main/java/io/questdb/cairo/wal/CopyWalSegmentUtils.java (2 hunks)
  • core/src/main/java/io/questdb/cairo/wal/WalPurgeJob.java (1 hunks)
  • core/src/main/java/io/questdb/cairo/wal/WalWriter.java (2 hunks)
  • core/src/main/java/io/questdb/cairo/wal/seq/TableSequencerAPI.java (1 hunks)
  • core/src/main/java/io/questdb/cairo/wal/seq/TableSequencerImpl.java (1 hunks)
  • core/src/main/java/io/questdb/cairo/wal/seq/TableTransactionLog.java (2 hunks)
  • core/src/main/java/io/questdb/cairo/wal/seq/TableTransactionLogV1.java (2 hunks)
  • core/src/main/java/io/questdb/cairo/wal/seq/TableTransactionLogV2.java (2 hunks)
  • core/src/main/java/io/questdb/cutlass/text/CopyTask.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/catalogue/WalTableListFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/catalogue/WalTransactionsFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/table/ReadParquetFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/table/parquet/MappedMemoryPartitionDescriptor.java (1 hunks)
  • core/src/main/java/io/questdb/std/Files.java (2 hunks)
  • core/src/main/java/io/questdb/std/FilesFacadeImpl.java (1 hunks)
  • core/src/main/java/io/questdb/std/MmapCache.java (6 hunks)
  • core/src/test/java/io/questdb/test/FilesTest.java (5 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • core/src/main/java/io/questdb/griffin/engine/table/parquet/MappedMemoryPartitionDescriptor.java
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (27)
  • GitHub Check: aarch64
  • GitHub Check: New pull request (SelfHosted Other tests on linux-arm64)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests on linux-x64-zfs)
  • GitHub Check: New pull request (SelfHosted Other tests on linux-x64-zfs)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests on linux-arm64)
  • GitHub Check: New pull request (SelfHosted Other tests Start ARM Agent)
  • GitHub Check: New pull request (SelfHosted Other tests Start X64Zfs Agent)
  • GitHub Check: New pull request (Hosted Running tests on windows-other)
  • GitHub Check: New pull request (Hosted Running tests on windows-pgwire)
  • GitHub Check: New pull request (Hosted Running tests on windows-cairo)
  • GitHub Check: New pull request (Hosted Running tests on windows-fuzz2)
  • GitHub Check: New pull request (Hosted Running tests on windows-fuzz1)
  • GitHub Check: New pull request (Hosted Running tests on windows-griffin)
  • GitHub Check: New pull request (Hosted Running tests on mac-other)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-other)
  • GitHub Check: New pull request (Hosted Running tests on mac-pgwire)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-pgwire)
  • GitHub Check: New pull request (Hosted Running tests on mac-cairo-fuzz)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-cairo)
  • GitHub Check: New pull request (Hosted Running tests on mac-cairo)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-fuzz)
  • GitHub Check: New pull request (Hosted Running tests on mac-griffin)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests Start X64Zfs Agent)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-griffin)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests Start ARM Agent)
  • GitHub Check: New pull request (Rust Test and Lint on linux-jdk17)
  • GitHub Check: New pull request (Check Changes Check changes)
🔇 Additional comments (39)
core/src/main/c/share/dedup.cpp (1)

635-643: Good fix: add explicit false return to close the non-null scan.

Prevents UB/compile error from falling off the end of a non-void lambda and aligns semantics with other is_not_null_lambda implementations in this file.

core/src/main/java/io/questdb/cutlass/text/CopyTask.java (1)

1342-1344: Guarded munmap looks good

Avoids invalid munmap on zero address.

core/src/main/java/io/questdb/griffin/engine/functions/table/ReadParquetFunctionFactory.java (1)

100-102: Guarded unmap in finally is correct

Prevents calling munmap on zero address.

.github/workflows/rebuild_native_libs.yml (1)

33-35: Thanks for linking the runner-images issue

The inline link addresses a prior nit about discoverability.

core/src/main/c/windows/files.c (1)

703-711: Windows mmap: POSIX-aligned handling of len==0

Early-return with ERROR_INVALID_PARAMETER for len==0 matches cross-platform behavior.

core/src/main/java/io/questdb/cairo/O3PartitionJob.java (2)

267-269: Guarded unmap in parquet read: good

Prevents invalid munmap calls.


2570-2572: Guarded unmap in index updater: good

Consistent with new munmap contract.

core/src/main/java/io/questdb/std/MmapCache.java (1)

287-289: LGTM: MAP_FAILED check for mmap accounting is consistent.

core/src/main/java/io/questdb/griffin/engine/functions/catalogue/WalTableListFunctionFactory.java (1)

316-320: LGTM: switched to isFileCannotRead() for clearer intent.

Consistent with the repo-wide move away from errno-based checks.

core/src/main/java/io/questdb/cairo/wal/seq/TableSequencerAPI.java (1)

152-161: LGTM: use isFileCannotRead()/isTableDropped() in catch.

Matches the new CairoException helpers and preserves previous behavior.

core/src/main/java/io/questdb/griffin/engine/functions/catalogue/WalTransactionsFunctionFactory.java (1)

118-118: Switch to CairoException.isFileCannotRead(): LGTM

Consistent with the PR-wide predicate rename; behavior preserved.

core/src/main/java/io/questdb/cairo/wal/seq/TableTransactionLogV1.java (1)

372-379: MAP_FAILED checks: LGTM

Using FilesFacade.MAP_FAILED for mmap/mremap failure detection aligns with cross-platform semantics introduced in this PR.

Also applies to: 391-395

core/src/main/java/io/questdb/cairo/wal/seq/TableSequencerImpl.java (2)

128-135: Open-path errno predicate migration: LGTM

Replacing errnoFileCannotRead() with isFileCannotRead() keeps the intended drop-detection logic intact.


120-141: No stale predicates or legacy mmap checks detected
Repository-wide scan found zero occurrences of errnoFileCannotRead or manual mmap/mremap comparisons against –1.

core/src/main/java/io/questdb/std/FilesFacadeImpl.java (1)

429-431: Rename-consistency: switched to Files.isErrnoFileDoesNotExist() — LGTM

Matches the new helper naming and preserves semantics.

core/src/main/java/io/questdb/cairo/TableNameRegistryStore.java (3)

359-365: Use of CairoException.isFileCannotRead(): correct behavior retained

Properly treats transient file-read/rename/drop scenarios as non-fatal during root dir scan.


433-441: RO-mode reload: correct switch to isFileCannotRead()

Keeps the RO retry/continue logic intact when tables.d.* is swapped by RW instance.


359-365: No remaining deprecated helper usages found
Ran the suggested RG scan across the codebase (excluding target/build); no occurrences of errnoFileCannotRead or errnoFileDoesNotExist detected.

core/src/main/java/io/questdb/cairo/wal/WalWriter.java (2)

1759-1766: Always unmap in finally: good catch

Moving munmap() to finally guarantees cleanup even on exceptions and aligns with stricter munmap preconditions.


1776-1783: Symmetric fix for data vector: LGTM

Same robustness applied to the data file path; matches the aux path change.

core/src/main/java/io/questdb/cairo/TableWriter.java (4)

5670-5673: Good: guarded munmap for parquet mapping.

The parquetAddr != 0 check prevents invalid unmap calls and matches the new stricter munmap semantics.


8556-8558: Good: parquet unmap guard.

Matches the new invariant and avoids spurious failures.


8668-8670: Good: parquet unmap guard here too.

Consistent with other call sites.


8738-8739: Verified API rename; no deprecated calls remain
rg confirms no errnoFileCannotRead() usages in TableWriter.java, and all ff.munmap() calls are properly guarded.

core/src/main/java/io/questdb/cairo/wal/seq/TableTransactionLog.java (1)

41-41: Import looks good

Os is used in this file; no action needed.

core/src/main/java/io/questdb/cairo/wal/CopyWalSegmentUtils.java (1)

27-43: Import changes are fine

Explicit imports improve clarity; no action needed.

core/src/main/java/io/questdb/cairo/CairoException.java (2)

45-47: Adding platform errno constants is correct

ERRNO_INVALID_PARAMETER and Windows equivalent are accurate and useful for cross-platform checks.


249-251: API rename aligns with Files helpers

isFileCannotRead() delegating to Files.isErrnoFileCannotRead(errno) is consistent with the rest of the PR.

core/src/main/java/io/questdb/cairo/wal/seq/TableTransactionLogV2.java (2)

486-493: Safe munmap guard in closePart()

Only unmapping when address > 0 matches the new failure semantics.


524-532: MAP_FAILED check and detailed exception are spot-on

Capturing mmap result, checking MAP_FAILED, and throwing with path and mapSize is correct and consistent.

core/src/main/java/io/questdb/cairo/TableUtils.java (3)

858-871: Switch to ex.isFileCannotRead() looks good

Matches the new CairoException API and Files helpers.


1191-1199: Use of MAP_FAILED sentinel is correct

Replacing addr > -1 with addr != FilesFacade.MAP_FAILED is the right failure check.


1297-1314: Error classification updated correctly

Switch to Files.isErrnoFileCannotRead(errno) keeps the logic consistent with renamed helpers.

core/src/main/java/io/questdb/std/Files.java (1)

128-131: OS-specific invalid-parameter errno helper: LGTM.

Clearer than sprinkling conditional constants at call sites.

core/src/test/java/io/questdb/test/FilesTest.java (5)

28-28: Import CairoException: LGTM.

Needed for the new exception-based mmap/mremap/munmap assertions.


65-65: Static import assertContains: LGTM.

Keeps intent concise in failure assertions.


830-839: Zero address munmap test: LGTM.

Accurately asserts the new guard behavior and message.


842-866: Zero length munmap test: LGTM.

Covers the “address or length” validation path and cleans up properly.


886-898: openCleanRW on directory negative FD test: LGTM.

Validates error propagation without leaking resources.

jerrinot
jerrinot previously approved these changes Sep 5, 2025
Copy link
Copy Markdown
Contributor

@jerrinot jerrinot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let the fuzzing commence!

@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 112 / 141 (79.43%)

file detail

path covered line new line coverage
🔵 io/questdb/cairo/wal/CheckWalTransactionsJob.java 0 1 00.00%
🔵 io/questdb/griffin/engine/functions/catalogue/WalTableListFunctionFactory.java 0 1 00.00%
🔵 io/questdb/cairo/wal/seq/TableSequencerAPI.java 0 1 00.00%
🔵 io/questdb/cairo/wal/seq/TableTransactionLog.java 3 8 37.50%
🔵 io/questdb/griffin/engine/functions/catalogue/WalTransactionsFunctionFactory.java 1 2 50.00%
🔵 io/questdb/std/Files.java 2 4 50.00%
🔵 io/questdb/cairo/TableNameRegistryStore.java 1 2 50.00%
🔵 io/questdb/cairo/wal/seq/TableTransactionLogV2.java 5 9 55.56%
🔵 io/questdb/cairo/wal/WalWriter.java 6 8 75.00%
🔵 io/questdb/PropServerConfiguration.java 18 22 81.82%
🔵 io/questdb/cairo/wal/CopyWalSegmentUtils.java 9 11 81.82%
🔵 io/questdb/cairo/TableUtils.java 6 7 85.71%
🔵 io/questdb/std/MmapCache.java 25 29 86.21%
🔵 io/questdb/cairo/wal/WalPurgeJob.java 1 1 100.00%
🔵 io/questdb/cairo/wal/seq/TableSequencerImpl.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/table/ReadParquetFunctionFactory.java 2 2 100.00%
🔵 io/questdb/cairo/O3PartitionJob.java 4 4 100.00%
🔵 io/questdb/std/FilesFacadeImpl.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/parquet/MappedMemoryPartitionDescriptor.java 10 10 100.00%
🔵 io/questdb/cairo/vm/MemoryPMARImpl.java 5 5 100.00%
🔵 io/questdb/cairo/TableWriter.java 7 7 100.00%
🔵 io/questdb/cairo/wal/seq/TableTransactionLogV1.java 2 2 100.00%
🔵 io/questdb/cutlass/text/CopyTask.java 2 2 100.00%
🔵 io/questdb/cairo/CairoException.java 1 1 100.00%

@puzpuzpuz puzpuzpuz self-requested a review September 5, 2025 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants