Repository navigation
Commit 284f211
fix: Remove
## Which issue does this PR close?
- Closes apache#25883.
## Rationale for this change
`datafusion-physical-plan` enables the `serde_json/preserve_order`
feature. Cargo feature unification turns this feature on for the whole
application that embeds DataFusion. Then every `serde_json::Map` in that
application is backed by `IndexMap` instead of `BTreeMap`. This changes
the key order of JSON that the application writes (REST responses,
signed JSON, snapshot tests), and the application did not ask for this
change.
DataFusion only needs a fixed key order for the `EXPLAIN FORMAT pgjson`
output (`"Node Type"` first, `"Plans"` last, as in PostgreSQL). A
library must not change a global serde behavior for this.
There is also a small latent bug: `datafusion-expr` enabled
`preserve_order` only as a dev-dependency. So in real builds, the key
order of the logical-plan pgjson output depended on whether some other
crate in the dependency graph enabled the feature.
## What changes are included in this PR?
- Physical plan pgjson (`datafusion/physical-plan/src/display.rs`): the
nodes are now `#[derive(Serialize)]` structs. The field order is the
output order. `Extras` is an ordered list of pairs that is serialized as
a map.
- Logical plan pgjson (`datafusion/expr/src/logical_plan/display.rs`):
there are many node shapes, so a small private `pg_fields!` macro builds
an ordered field list. It replaces the `json!({...})` calls almost
one-for-one. A short manual `Serialize` impl writes the fields in order.
- Remove `features = ["preserve_order"]` from `datafusion-physical-plan`
and from the `datafusion-expr` dev-dependencies. `datafusion-cli` and
`datafusion-sqllogictest` keep the feature, because they are binaries or
test crates.
- Add `serde` (with `derive`) as a workspace dependency. It was already
in the dependency tree through `serde_json`.
`cargo tree -e features -i serde_json -p datafusion` now shows no
`preserve_order`.
## What is the testing strategy for this PR?
The existing tests cover this change, and the output is byte-identical:
- `pgjson_snapshot_of_sample_plan` in `datafusion-physical-plan` and
`test_display_pg_json` in `datafusion-expr`. These tests now build
without `preserve_order`, so they really check the key order. With
sorted keys, `"Details"` would come before `"Node Type"` and the
snapshot would fail.
- `explain.slt` and `explain_analyze.slt` pass.
## Are there any user-facing changes?
- Applications that depend on DataFusion no longer get
`serde_json/preserve_order` from DataFusion.
- The `EXPLAIN FORMAT pgjson` output does not change, with one small
exception. With `ShowMetrics::Full`, `"Actual Rows"` now always comes
before `"Actual Total Time"`. Before, the order followed the order in
which the operator registered its metrics.
- Note: the third-party `substrait` crate also enables
`serde_json/preserve_order`. Users of `datafusion-substrait` still get
this feature from that crate.
No public API changes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <[email protected]>serde_json/preserve_order feature from library crates (apache#25884)1 parent afe4a63 commit 284f211
6 files changed
Lines changed: 240 additions & 146 deletions
File tree
- datafusion
- expr
- src/logical_plan
- physical-plan
- src
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
197 | 197 | | |
198 | 198 | | |
199 | 199 | | |
| 200 | + | |
200 | 201 | | |
201 | 202 | | |
202 | 203 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
| 68 | + | |
68 | 69 | | |
69 | 70 | | |
70 | 71 | | |
| |||
73 | 74 | | |
74 | 75 | | |
75 | 76 | | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | 77 | | |
80 | 78 | | |
81 | 79 | | |
| |||
0 commit comments