Skip to content

[arrow-avro] suggest switching from xz to liblzma #8970

@getChan

Description

@getChan

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

It seems that xz is a crate that is no longer maintained. How about switching to liblzma, which is being maintained as a fork?

related issue apache/datafusion#15342 (comment)

Describe the solution you'd like

Describe alternatives you've considered

Additional context

When importing arrow-avro from an external source that depends on liblzma, duplicate dependencies occur in the form of xz -> lzma-sys / liblzma -> lzma-sys.

stdout: error: failed to select a version for `lzma-sys`.
    ... required by package `xz2 v0.1.0`
    ... which satisfies dependency `xz2 = "^0.1"` of package `xz v0.1.0`
    ... which satisfies dependency `xz = "^0.1"` of package `arrow-avro v57.1.0`
    ... which satisfies dependency `arrow-avro = "^57.0.0"` (locked to 57.1.0) of package `datafusion-datasource-avro v51.0.0 (/Users/t1100490/chan/repos/datafusion/datafusion/datasource-avro)`
    ... which satisfies path dependency `datafusion-datasource-avro` (locked to 51.0.0) of package `datafusion v51.0.0 (/Users/t1100490/chan/repos/datafusion/datafusion/core)`
    ... which satisfies path dependency `datafusion` (locked to 51.0.0) of package `datafusion-benchmarks v51.0.0 (/Users/t1100490/chan/repos/datafusion/benchmarks)`
versions that meet the requirements `^0.1` are: 0.1.20, 0.1.19, 0.1.17, 0.1.16, 0.1.15, 0.1.14, 0.1.13, 0.1.12, 0.1.11, 0.1.10, 0.1.9, 0.1.8, 0.1.7, 0.1.6, 0.1.5, 0.1.4, 0.1.3, 0.1.2, 0.1.1, 0.1.0

package `lzma-sys` links to the native library `lzma`, but it conflicts with a previous package which links to `lzma` as well:
package `liblzma-sys v0.4.4`
    ... which satisfies dependency `liblzma-sys = "^0.4.3"` (locked to 0.4.4) of package `liblzma v0.4.5`
    ... which satisfies dependency `liblzma = "^0.4.4"` (locked to 0.4.5) of package `datafusion v51.0.0 (/Users/t1100490/chan/repos/datafusion/datafusion/core)`
    ... which satisfies path dependency `datafusion` (locked to 51.0.0) of package `datafusion-benchmarks v51.0.0 (/Users/t1100490/chan/repos/datafusion/benchmarks)`
Only one package in the dependency graph may specify the same links value. This helps ensure that only one copy of a native library is linked in the final binary. Try to adjust your dependencies so that only one package uses the `links = "lzma"` value. For more information, see https://doc.rust-lang.org/cargo/reference/resolver.html#links.

Metadata

Metadata

Assignees

Labels

arrowChanges to the arrow cratearrow-avroarrow-avro crateenhancementAny new improvement worthy of a entry in the changelog

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions