Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

@coderabbitai: please review, pay attention to the CMake script setup, pull enough context to see the big picture

#!/bin/bash

# Get PR details and list of changed files
echo "=== PR Details ==="
gh pr view 6477 --json title,body,files --jq '.title, .body, .files[].path'

echo -e "\n=== Changed Files ==="
gh pr diff 6477 --name-only

echo -e "\n=== Full PR Diff ==="
gh pr diff 6477

#!/bin/bash

# Look for more CMake context and build system files
echo "=== CMake files in the repository ==="
fd -e cmake -e txt CMakeLists

echo -e "\n=== Looking for jemalloc references in CMake files ==="
rg -n -C3 "jemalloc" --type=cmake

echo -e "\n=== Check main/root CMakeLists.txt if it exists ==="
fd -t f "^CMakeLists.txt$" --max-depth 3 --exec head -100 {}

echo -e "\n=== Look for build scripts ==="
fd -e sh -e py build --max-depth 2

echo -e "\n=== Check for jemalloc usage in Java/C code ==="
rg -n -C2 "LD_PRELOAD|jemalloc" --type=java --type=c --type=cpp -g '!*.so' -g '!*.dylib'

echo -e "\n=== Check the complete jemalloc-cmake directory structure ==="
fd . "core/src/main/c/share/jemalloc-cmake" --type f --exec echo "File: {}" \; --exec head -30 {}

#!/bin/bash

# Check the symbols exported by the new jemalloc binaries
echo "=== Checking exported symbols in Linux x86_64 libjemalloc.so ==="
nm -D core/src/main/bin/linux-x86-64/libjemalloc.so 2>/dev/null | grep -E "malloc|free|calloc|realloc" | head -20

echo -e "\n=== Check if je_ prefixed symbols still exist ==="
nm -D core/src/main/bin/linux-x86-64/libjemalloc.so 2>/dev/null | grep "je_malloc" | head -10

echo -e "\n=== Look for how os.c uses jemalloc ==="
cat core/src/main/c/share/os.c | grep -A10 -B10 "jemalloc"

echo -e "\n=== Check JEMALLOC_NO_PRIVATE_NAMESPACE usage ==="
rg -n "JEMALLOC_NO_PRIVATE_NAMESPACE" -C2

echo -e "\n=== Understand the CMake flow - see how jemalloc links to main library ==="
rg -n "target_link_libraries.*jemalloc" --type=cmake -C2

#!/bin/bash

# Get more context about the build system and jemalloc integration
echo "=== Check how questdb library links with jemalloc ==="
rg -n "jemalloc" core/CMakeLists.txt -C5

echo -e "\n=== Look at the complete jemalloc CMakeLists.txt ==="
cat core/src/main/c/share/jemalloc-cmake/CMakeLists.txt

echo -e "\n=== Check what changed in the header file - show the actual macro definitions ==="
echo "--- Lines 70-100 of jemalloc.h (the macro section) ---"
sed -n '70,100p' core/src/main/c/share/jemalloc-cmake/include/jemalloc/jemalloc.h

echo -e "\n=== Compare Darwin vs Linux configs for JEMALLOC_IS_MALLOC ==="
echo "Darwin x86_64:"
grep -A2 "JEMALLOC_IS_MALLOC" core/src/main/c/share/jemalloc-cmake/include_darwin_x86_64/jemalloc/internal/jemalloc_internal_defs.h.in
echo -e "\nLinux x86_64:"
grep -A2 "JEMALLOC_IS_MALLOC" core/src/main/c/share/jemalloc-cmake/include_linux_x86_64/jemalloc/internal/jemalloc_internal_defs.h.in

echo -e "\n=== Check if there's documentation about jemalloc usage ==="
fd -e md README -x grep -l "jemalloc" {}

@jerrinot: I've reviewed the jemalloc integration fix with attention to the CMake setup and overall architecture. Here's my comprehensive analysis:

Overview - The Big Picture

This PR fixes a critical issue where jemalloc was built with symbol prefixing (je_malloc, je_free, etc.), preventing LD_PRELOAD from working correctly. When the jemalloc library was preloaded, applications continued using the system allocator because they expected standard malloc/free symbols.

CMake Architecture

Build System Structure:

• core/CMakeLists.txt includes jemalloc-cmake/CMakeLists.txt at line 368
• Platform-specific headers are organized in directories: include_{os}_{arch}/ (e.g., include_linux_x86_64)
• CMake selects the appropriate header directory based on OS_LINUX/OS_DARWIN/OS_FREEBSD and ARCH_AMD64/ARCH_AARCH64 flags
• The configure_file() command templates jemalloc_internal_defs.h.in → jemalloc_internal_defs.h, substituting @JEMALLOC_CONFIG_MALLOC_CONF@ with the actual configuration string

Configuration Management:

set(JEMALLOC_CONFIG_MALLOC_CONF "percpu_arena:percpu,oversize_threshold:0,muzzy_decay_ms:0,dirty_decay_ms:5000,background_thread:true")

This is templated into the binary at build time, and can be overridden via JEMALLOC_CONFIG_MALLOC_CONF_OVERRIDE cache variable.

Key Changes

1. Symbol Export Fix (Core Issue)

In jemalloc.h (lines 73-98):

// BEFORE: #define je_malloc je_malloc
// AFTER:  #define je_malloc malloc

This changes how jemalloc's internal je_malloc function is exposed - now it's exported as standard malloc.

2. JEMALLOC_IS_MALLOC Flag

In Linux platform configs:

// BEFORE: /* #define JEMALLOC_IS_MALLOC */
// AFTER:  #define JEMALLOC_IS_MALLOC

This flag tells jemalloc to take ownership of standard allocator symbol names. Critically, this is only enabled for:

• ✅ Linux (x86_64 and aarch64)
• ✅ FreeBSD (x86_64 and aarch64)
• ❌ Darwin/macOS (intentionally left disabled)

The Darwin exclusion makes sense because macOS uses zone allocators and has different dynamic linking semantics.

3. Background Thread Enhancement

Added background_thread:true to the Linux malloc configuration. This enables jemalloc's background thread for asynchronous memory management operations, which can reduce latency spikes from foreground thread memory operations.

4. JEMALLOC_NO_PRIVATE_NAMESPACE

The CMake sets -DJEMALLOC_NO_PRIVATE_NAMESPACE, which prevents internal jemalloc symbols from being prefixed, keeping only public API symbols managed through the macro system.

Architecture Strengths

1. Platform Isolation: Each OS/architecture combination has its own configuration, allowing fine-grained control
2. Build-time Configuration: The malloc configuration is baked into the binary, avoiding runtime parsing overhead
3. Override Capability: The JEMALLOC_CONFIG_MALLOC_CONF_OVERRIDE cache variable allows customization without modifying source
4. Proper Symbol Visibility: The combination of JEMALLOC_NO_PRIVATE_NAMESPACE and JEMALLOC_IS_MALLOC provides clean symbol exports

Verification Points

The binary changes are consistent:

• All 4 jemalloc libraries rebuilt (darwin-aarch64/x86-64, linux-aarch64/x86-64)
• libquestdb.dylib and libquestdb.dll also rebuilt (likely link against jemalloc)

Potential Considerations

1. Symbol Conflicts: Now that jemalloc exports standard malloc/free symbols, if another library in the same process also tries to export these symbols, you could have conflicts. This is generally fine for QuestDB's architecture, but worth noting.

2. Background Thread Impact: The background_thread:true addition will create an additional thread per arena. This is generally beneficial, but increases thread count. The configuration comment mentions this is inherited from ClickHouse's experience.

3. macOS LD_PRELOAD: The fix doesn't enable JEMALLOC_IS_MALLOC on macOS, so DYLD_INSERT_LIBRARIES (macOS equivalent of LD_PRELOAD) still won't use jemalloc. This appears intentional given macOS's zone allocator architecture.

Summary

This is a well-architected fix. The CMake setup properly:

• ✅ Manages platform-specific configurations
• ✅ Templates build-time settings into binaries
• ✅ Provides clean symbol exports for LD_PRELOAD compatibility
• ✅ Maintains platform differences (Linux vs. macOS)
• ✅ Adds performance enhancement (background_thread) along with the fix

The changes correctly address the root cause: jemalloc now exports standard allocator symbols on Linux, making LD_PRELOAD functional as intended.

@puzpuzpuz good point. I removed macos support entirely, and change questdb.sh to fail-fast when you attempt to enable JEMALLOC on an unsupported platform

@puzpuzpuz thanks for your help with this PR!

@puzpuzpuz thanks for your help with this PR!

Thanks for finding that we lack the symbols in the binary and fixing it.

@jerrinot how about adding a CI Linux runner that would use jemalloc?

@jerrinot do you need some help with this one? I'm interested in getting it shipped in the next release.

@puzpuzpuz thanks for the ping. I pushed changes. I added jemalloc to coverage tests. since these are the only tests running in hotspot@x86_64
let' see.

@puzpuzpuz thanks for the ping. I pushed changes. I added jemalloc to coverage tests. since these are the only tests running in hotspot@x86_64

Thanks! How about enabling jemalloc by default on Linux in questdb.sh?

@puzpuzpuz done!

[PR Coverage check]

😍 pass : 1 / 1 (100.00%)

file detail