JVM crash Session causes SIGSEGV/SIGBUS under repeated use of Session.create()

What happened?

There is a reproducer of the issue in
https://github.com/dfa1/datastore — a benchmark comparing CSV / JSON / Parquet / Vortex storage formats.

  git clone https://github.com/dfa1/datastore.git
  cd datastore
  mvn test -Dtest="ReadColumnTest,VortexOhlcStoreTest"  # passes (few iterations)
  mvn test -Dtest="BenchmarkTest"                        # crashes (84+ Vortex ops)

BenchmarkTest runs 7 scales × (2 warmup + 10 measured) iterations × 3 operations (write / read /
readColumn) = 252 Session objects created in one JVM process.

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00000001393da064, pid=71272, tid=5635
#
# JRE version: OpenJDK Runtime Environment (27.0+1) (build 27-jep401ea3+1-1)
# Java VM: OpenJDK 64-Bit Server VM (27-jep401ea3+1-1, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-aarch64)
# Problematic frame:
# C  [libvortex_jni9112159322116037794.dylib+0x32064]  vortex_file::writer::WriteOptionsSessionExt::write_options::h986f424532314d6f+0x18
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again

Possible fix:

Make Session implement AutoCloseable and expose the Cleanable so callers can release native deterministically:

  public final class Session implements AutoCloseable {
      private final Cleaner.Cleanable cleanable;

      private Session(long pointer) {
          Preconditions.checkArgument(pointer != 0, "invalid session pointer");
          this.pointer   = pointer;
          this.cleanable = VortexCleaner.register(this, () -> NativeSession.free(pointer));
      }

      @Override
      public void close() {
          cleanable.clean(); // idempotent — safe to call; Cleaner won't double-free
      }
  }

Callers could then write:

  try (Session session = Session.create()) {
      // ...
  }

Steps to reproduce

see the attached instructions above

Environment

vortex-jni version: 0.72.0
JDK 27 EA (Homebrew, aarch64 macOS): crashes ✗ — SIGSEGV/SIGBUS under repeated Session use
Arch: bsd-aarch64

Additional context

I got the error also once in JDK 25 but it is much harder to reproduce.
The proposed fix is the idiomatic way in Java to deal with external resources.

NativeSession.free is also exposed as public method: I tried to use it by I got a double free (because free is also called by the cleaner).
IMHO it should be marked as package private (not part of the public API at least)

hs_err_pid66301.log
Here is the full log, interesting part is:

Current thread (0x0000000104f3cfd0):  JavaThread "main"             [_thread_in_native, id=6147, stack(0x000000016b624000,0x000000016b827000) (2060K)]

Stack: [0x000000016b624000,0x000000016b827000],  sp=0x000000016b822040,  free space=2040k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libvortex_jni5295561805888454796.dylib+0x32064]  vortex_file::writer::WriteOptionsSessionExt::write_options::h986f424532314d6f+0x18
C  [libvortex_jni5295561805888454796.dylib+0x264d0]  vortex_jni::writer::Java_dev_vortex_jni_NativeWriter_create::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h84fc0ede4f6ff94b+0xb4c
C  [libvortex_jni5295561805888454796.dylib+0x71c70]  _$LT$tracing..instrument..Instrumented$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::he6d38967d6ba0ad1+0xf0
C  [libvortex_jni5295561805888454796.dylib+0x55e658]  async_task::raw::RawTask$LT$F$C$T$C$S$C$M$GT$::run::h7f6d87ee8210d49c+0x88
C  [libvortex_jni5295561805888454796.dylib+0x135b8]  Java_dev_vortex_jni_NativeWriter_close+0x698
j  dev.vortex.jni.NativeWriter.close(J)V+0
J 19933 c2 io.github.dfa1.datastore.VortexOhlcStore.write(Ljava/util/stream/Stream;Ljava/nio/file/Path;)V (170 bytes) @ 0x000000011c0c2914 [0x000000011c0c1d00+0x0000000000000c14]
j  io.github.dfa1.datastore.BenchmarkTest.measure(Lio/github/dfa1/datastore/OhlcStore;Ljava/util/List;Ljava/nio/file/Path;)Lio/github/dfa1/datastore/BenchmarkTest$Result;+110
j  io.github.dfa1.datastore.BenchmarkTest.lambda$storageComparison$0(Ljava/util/List;Ljava/nio/file/Path;Lio/github/dfa1/datastore/OhlcStore;)Lio/github/dfa1/datastore/BenchmarkTest$Result;+4

More minimal reproducer: this fails after 200-300 iterations on JDK 26 and about 18K on JDK 25

	@Test
	void reproducer(@TempDir Path dir) throws IOException {
		VortexOhlcStore store = new VortexOhlcStore();
		int i = 0;
		while (true) {
			System.out.println(i);
			OhlcGenerator aaa = new OhlcGenerator(new Symbol("AAA"), LocalDate.now(), 10, 10);
			store.write(aaa.stream(10), dir.resolve("a" + i++));
		}
	}

test: add minimal reproducer for vortex-data/vortex#8075

test: exclude diag tests from default run; disable CrashTest

@dfa1

Hey @dfa1, I was able to reproduce the failure from your last example. Thanks for including a compact repro.

Interestingly, moving the Session object in the OhlcStore write method to be a static or instance-var prevents the failure.

I agree that the Session should be AutoCloseable.

Separately, I'd recommend that you update the Vortex store to hold a long-lived Session object. Creating new Sessions can be expensive, generally they should be initialized statically and shared across a process.

test(vortex): add repro for shared Session errno=22 on larger re-read

@a10y

Thanks @a10y for feedback! I moved Session to a private static final field so it's created once and shared across all operations.

This fixes the crash I reported, but I'm now seeing a different failure when the same session is reused across multiple sequential writes + reads on different file paths:

java.io.IOException: [errno 22] CDataJniException{errno=22, message=External error: SerializedArray

buffer is too short for flatbuffer}
at org.apache.arrow.c.ArrowArrayStream.getNext(ArrowArrayStream.java:146)
at org.apache.arrow.c.ArrowArrayStreamReader.loadNextBatch(ArrowArrayStreamReader.java:63)

This was the reason why I started to create a new Session per call.

Repro pattern (no concurrency, single thread):

    private static final Session SESSION = Session.create();

    // iteration 1: ~250 rows → succeeds
    VortexWriter.create(SESSION, uri1, schema, opts, allocator);  // write
    DataSource.open(SESSION, uri1).scan(...);                      // read OK

    // iteration 2: ~2500 rows → read throws errno=22
    VortexWriter.create(SESSION, uri2, schema, opts, allocator);  // write
    DataSource.open(SESSION, uri2).scan(...);                      // FAIL

The first iteration (small file) succeeds. The second iteration (larger file, different URI) fails
during loadNextBatch. There is a new reproducer here:

https://github.com/dfa1/datastore/blob/main/src/test/java/io/github/dfa1/datastore/VortexSessionReuseReproTest.java

Perhaps I'm using the API in the wrong way?

Ahh ok, I see this now. So the issue is that the Session under the hood caches footers/metadata by the URI. It's meant for Object storage, so if you write a file, read it, then rewrite the same file path with different data and try and read again via the same session, you'll hit this issue.

Reason for this is the most common deployment environment is where you are using S3 and the files are immutable after they are written.

The immediate fix is to write to different tempfiles for your tests.

In Vortex, we should probably document this behavior in the Java docs (it's documented in the Rust API I believe), and also make it possible to disable the cache

that explains everything, thank you! I'm going to adapt my code but should I keep this issue open then?

Yep leave it open for now. We can use it to track the docs/config

Navigation Menu

Repository navigation

JVM crash Session causes SIGSEGV/SIGBUS under repeated use of Session.create() #8075

What happened?

Steps to reproduce

Environment

Additional context

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

JVM crash Session causes SIGSEGV/SIGBUS under repeated use of Session.create() #8075

Description

What happened?

Steps to reproduce

Environment

Additional context

Activity

dfa1 commented on May 24, 2026

dfa1 commented on May 24, 2026

dfa1 commented on May 24, 2026

a10y commented on May 26, 2026

dfa1 commented on May 26, 2026

a10y commented on May 26, 2026

dfa1 commented on May 26, 2026

a10y commented on May 26, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions