Skip to content

DFLib / Parquet and Avro classpath issues #30

@andrus

Description

@andrus

Another classpath issue potentially related to Ivy. We can't work with Parquet or Avro APIs. It works well in a standalone Java app, as well as in JShell (with /env --class-path taken from the Java app). But in Jupyter (jjava M2) the code fails with NoClassDefFoundError.

I vaguely remember this happening with Avro back in the iJava days as well, though we ignored it cause we could not do anything about it.

Parquet

%maven org.dflib:dflib-jupyter:1.0.0-SNAPSHOT

// the error is the same with or without this dependency
// %maven com.fasterxml.woodstox:woodstox-core:5.4.0

import org.dflib.parquet.*;

Parquet.loader().load("some.parquet");
java.lang.NoClassDefFoundError: com/ctc/wstx/io/InputBootstrapper
	at org.apache.parquet.conf.HadoopParquetConfiguration.<init>(HadoopParquetConfiguration.java:39)
	at org.apache.parquet.conf.HadoopParquetConfiguration.<init>(HadoopParquetConfiguration.java:35)
	at org.apache.parquet.hadoop.ParquetReader$Builder.<init>(ParquetReader.java:236)
	at org.dflib.parquet.read.DataframeParquetReaderBuilder.<init>(DataframeParquetReaderBuilder.java:10)
	at org.dflib.parquet.ParquetLoader.load(ParquetLoader.java:48)
	at org.dflib.parquet.ParquetLoader.load(ParquetLoader.java:28)
	at org.dflib.parquet.ParquetLoader.load(ParquetLoader.java:32)
	at .(#24:1)

Avro

Avro.loader().load("some.avro")
java.lang.NoClassDefFoundError: org/apache/avro/LogicalTypes$LogicalTypeFactory
	at .(#21:1)

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions