Skip to content

Commit 6d3321d

Browse files
committed
feat: add optional JSON functions crate with json_get_str scaffolding
Adds a new `datafusion-functions-json` crate to the workspace with `json_get_str` as the initial function, establishing the integration pattern for bringing JSON functions into core DataFusion. This PR implements the scaffolding discussed in apache#21301: - New crate `datafusion/functions-json/` with registration hook - `json_get_str(json, *keys) -> str` extracts a string value from a JSON string at the given path (supports nested keys and array indices) - Optional feature flag `json_expressions` on the core crate (not default) - Handles Utf8, LargeUtf8, and Utf8View string types - Minimum 2 args enforced at planning time via coerce_types - SQL-level tests via sqllogictest - 9 unit tests + 1 doctest Follow-on PRs will add the remaining functions from datafusion-functions-json (json_get, json_get_int, json_get_float, json_get_bool, json_get_json, json_get_array, json_as_text, json_length, json_contains, and -> / ->> operators). Closes apache#21301 (partial)
1 parent 0838a4d commit 6d3321d

8 files changed

Lines changed: 766 additions & 0 deletions

File tree

‎Cargo.lock‎

Lines changed: 16 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ members = [
3434
"datafusion/ffi",
3535
"datafusion/functions",
3636
"datafusion/functions-aggregate",
37+
"datafusion/functions-json",
3738
"datafusion/functions-aggregate-common",
3839
"datafusion/functions-table",
3940
"datafusion/functions-nested",
@@ -140,6 +141,7 @@ datafusion-ffi = { path = "datafusion/ffi", version = "54.0.0" }
140141
datafusion-functions = { path = "datafusion/functions", version = "54.0.0" }
141142
datafusion-functions-aggregate = { path = "datafusion/functions-aggregate", version = "54.0.0" }
142143
datafusion-functions-aggregate-common = { path = "datafusion/functions-aggregate-common", version = "54.0.0" }
144+
datafusion-functions-json = { path = "datafusion/functions-json", version = "54.0.0" }
143145
datafusion-functions-nested = { path = "datafusion/functions-nested", version = "54.0.0", default-features = false }
144146
datafusion-functions-table = { path = "datafusion/functions-table", version = "54.0.0" }
145147
datafusion-functions-window = { path = "datafusion/functions-window", version = "54.0.0" }
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
[package]
19+
name = "datafusion-functions-json"
20+
description = "JSON function package for the DataFusion query engine"
21+
keywords = ["datafusion", "json", "functions"]
22+
readme = "README.md"
23+
version = { workspace = true }
24+
edition = { workspace = true }
25+
homepage = { workspace = true }
26+
repository = { workspace = true }
27+
license = { workspace = true }
28+
authors = { workspace = true }
29+
rust-version = { workspace = true }
30+
31+
[package.metadata.docs.rs]
32+
all-features = true
33+
34+
# Note: add additional linter rules in lib.rs.
35+
# Rust does not support workspace + new linter rules in subcrates yet
36+
# https://github.com/rust-lang/cargo/issues/13157
37+
[lints]
38+
workspace = true
39+
40+
[lib]
41+
name = "datafusion_functions_json"
42+
43+
[features]
44+
default = []
45+
46+
[dependencies]
47+
arrow = { workspace = true }
48+
datafusion-common = { workspace = true }
49+
datafusion-doc = { workspace = true }
50+
datafusion-execution = { workspace = true }
51+
datafusion-expr = { workspace = true, default-features = false }
52+
datafusion-macros = { workspace = true }
53+
log = { workspace = true }
54+
serde_json = { workspace = true }
55+
56+
[dev-dependencies]
57+
arrow = { workspace = true, features = ["test_utils"] }
58+
tokio = { workspace = true, features = ["macros", "rt"] }
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
<!---
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
-->
19+
20+
# datafusion-functions-json
21+
22+
JSON scalar functions for [DataFusion](https://datafusion.apache.org/).
23+
24+
This crate provides JSON manipulation functions operating on JSON-encoded strings.
25+
Based on the [datafusion-functions-json](https://github.com/datafusion-contrib/datafusion-functions-json)
26+
community crate.
27+
28+
## Functions
29+
30+
| Function | Description |
31+
|----------|-------------|
32+
| `json_get_str(json, key1, ...)` | Extract a string value from a JSON string at the given path |
33+
34+
## Usage
35+
36+
These functions are registered automatically when the `json_expressions` feature
37+
is enabled on the `datafusion` crate. They can also be registered manually:
38+
39+
```rust
40+
use datafusion_functions_json;
41+
// registry is a FunctionRegistry, e.g. SessionState
42+
datafusion_functions_json::register_all(&mut registry)?;
43+
```

0 commit comments

Comments
 (0)