Skip to content

feat: add --json-path flag to navigate nested JSON arrays for input #141

Description

@vmvarela

Description

When using `-I json`, the parser currently requires the input to be a top-level array of objects:

[{"name": "Alice", "age": 30}, {"name": "Bob", "age": 25}]

Real-world APIs almost always wrap their data inside a key:

{"data": [{"name": "Alice"}, {"name": "Bob"}]}
{"results": {"items": [{"id": 1, "title": "..."}]}}
{"feed": {"channel": {"item": [{"title": "..."}]}}}

Today these all fail with: `JSON input must be an array of objects`.

This is the JSON equivalent of the bug fixed in #139 for XML (`--xml-root` / `--xml-row`).

Proposed flag

# dot-separated path to the key containing the row array
sql-pipe -I json --json-path data \
  'SELECT name, age FROM t'

sql-pipe -I json --json-path results.items \
  'SELECT id, title FROM t LIMIT 10'

# Real-world example — GitHub API response
curl https://api.github.com/repos/vmvarela/sql-pipe/issues \
  | sql-pipe -I json 'SELECT number, title FROM t WHERE state = "open"'
# (already works — GH returns a bare array)

curl "https://api.example.com/feed" \
  | sql-pipe -I json --json-path feed.entries \
    'SELECT title, published FROM t ORDER BY published DESC LIMIT 5'

Implementation sketch

  1. src/main.zig — add `--json-path <dot.separated.key>` flag; store in `ParsedArgs`, `ColumnsArgs`, `ValidateArgs`
  2. src/json.zig — `loadJsonArray` — accept `?[]const u8` path; after parsing, walk the segments (split on `.`) to reach the target value; verify it is an array, then proceed as today
  3. `--columns` and `--validate` modes — also forward the flag so schema inspection works on nested documents
  4. Tests — add integration tests for single-key path, multi-segment path, missing key error, non-array target error

Behaviour when flag is omitted

No change — `null` path means expect a bare top-level array (existing behaviour preserved).

Notes

  • NDJSON is line-oriented; nested navigation is less useful there but a `--ndjson-key` variant could unwrap a single wrapper key per line (lower priority, can be done in a follow-up)
  • Path syntax: dot-separated keys only for now (no array index support like `data[0].items`)
  • Key lookup is case-sensitive (consistent with the rest of the JSON parser)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions