Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: vmvarela/sql-pipe
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.8.1
Choose a base ref
...
head repository: vmvarela/sql-pipe
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v0.9.0
Choose a head ref
  • 8 commits
  • 14 files changed
  • 1 contributor

Commits on May 8, 2026

  1. refactor: modularize source — format, args, sqlite, loader, modes (#146)

    * feat: introduce format.zig with InputFormat, OutputFormat, and OutputWriter
    
    Extracts InputFormat and OutputFormat enums (with parse() methods) and the
    writeField CSV helper from main.zig into a new src/format.zig module.
    Introduces OutputWriter, a stateful struct that dispatches output formatting
    across all five formats (csv, tsv, json, ndjson, xml), eliminating the
    format-switch inside execQuery. Closes no issue yet; part of #145.
    
    * refactor: extract CLI argument types and parseArgs into args.zig
    
    * refactor: consolidate SQLite helpers and ColumnType into sqlite.zig
    
    * refactor: extract CSV loader and type inference into loader.zig
    
    * refactor: extract runColumns, runValidate, runSample into src/modes/
    
    * fix: address code review — consolidate ExitCode, remove dead code, guard XML validation
    vmvarela authored May 8, 2026
    Configuration menu
    Copy the full SHA
    1e7a70a View commit details
    Browse the repository at this point in the history

Commits on May 10, 2026

  1. refactor: eliminate code duplication and simplify core patterns (#147)

    - Remove 4 identical copies of fatal() — all files now import the pub
      fn from sqlite.zig
    - Collapse 19-case error switch in main() from ~100 lines to 22 using
      fatal() directly (it is noreturn)
    - Replace if-chain parse() in InputFormat/OutputFormat with
      std.meta.stringToEnum — stays in sync with enum variants automatically
    - Use beginTransaction/commitTransaction helpers in loader.zig instead
      of duplicating the inline sqlite3_exec logic
    - Replace byte-by-byte takeByte() JSON reading in columns.zig and
      validate.zig with allocRemaining(.unlimited)
    - Remove redundant @as(usize, N) casts in csv.zig unit tests
    
    No behaviour change. All tests pass, ziglint clean.
    vmvarela authored May 10, 2026
    Configuration menu
    Copy the full SHA
    216ecfc View commit details
    Browse the repository at this point in the history
  2. feat: disk-backed large dataset support (--disk flag) (#148)

    * feat: add --disk flag for file-backed large dataset support
    
    Switch from :memory: to a SQLite private temp file when --disk is set.
    Also enables PRAGMA temp_store = FILE so ORDER BY/GROUP BY transient
    structures spill to disk. Closes #91.
    
    * fix: address review findings on --disk PR
    
    - Update loop invariant comment to include disk flag
    - Add explanatory comments on tests 125-127 (TEXT→NUMERIC SQLite conversion)
    - Add tests 129-131: --disk + --no-type-inference, --header, --output
    vmvarela authored May 10, 2026
    Configuration menu
    Copy the full SHA
    9010bbd View commit details
    Browse the repository at this point in the history
  3. chore: Update version number in README installation instructions

    Updated version number in installation instructions for Debian, RPM, and APK.
    vmvarela authored May 10, 2026
    Configuration menu
    Copy the full SHA
    66f2dc4 View commit details
    Browse the repository at this point in the history
  4. fix(xml): verify nested closing tag names in readContent (#149)

    * fix(xml): verify nested closing tag names in readContent
    
    Maintain a tag stack inside XmlParser.readContent so that every
    closing tag of a nested element is validated against the name of
    the element that opened it.  Previously the name was read and
    silently discarded, accepting malformed XML like
    <col><a>text</b></col> without error.
    
    Closes #134
    
    * fix(xml): add safety comment on pop().? in readContent
    vmvarela authored May 10, 2026
    Configuration menu
    Copy the full SHA
    89ad439 View commit details
    Browse the repository at this point in the history
  5. perf: replace byte-by-byte takeByte() with block reads in xml.zig and…

    … json.zig (#150)
    
    Replace the tight takeByte() loops that read stdin one byte at a time
    with reader.allocRemaining(allocator, .unlimited), which issues a single
    bulk read using the Io vtable dispatch instead of one dispatch per byte.
    
    Functions updated:
      - xml.zig: loadXmlInput, getXmlColumnNames, summarizeXml
      - json.zig: loadJsonArray
    
    No API changes; all existing tests continue to pass.
    
    Closes #135
    vmvarela authored May 10, 2026
    Configuration menu
    Copy the full SHA
    27a85bd View commit details
    Browse the repository at this point in the history
  6. feat: add --json-path flag to navigate nested JSON arrays for input (#…

    …151)
    
    * feat: add --json-path flag to navigate nested JSON arrays for input
    
    Add --json-path <dot.separated.path> flag for -I json mode, allowing
    users to reach a nested array inside a JSON document:
    
      curl https://api.example.com/feed     | sql-pipe -I json --json-path results.items 'SELECT title FROM t'
    
    The flag is also forwarded to --columns and --validate so schema
    inspection works on nested documents.  Omitting the flag preserves
    existing behaviour (top-level bare array expected).
    
    Adds navigateJsonPath() helper in json.zig, threads json_path through
    ParsedArgs / ColumnsArgs / ValidateArgs, and adds 5 integration tests
    (single-key path, multi-segment path, missing key, non-array target,
    --columns with --json-path).
    
    Closes #141
    
    * fix: validate --json-path requires -I json; add validate test; renumber tests 132→133
    vmvarela authored May 10, 2026
    Configuration menu
    Copy the full SHA
    4e94a6c View commit details
    Browse the repository at this point in the history
  7. docs: add XML file usage examples to usage, man page, and README (#152)

    Add --xml-root / --xml-row flag examples in all three documentation
    locations so users can discover XML input support for nested documents
    like RSS feeds:
    
    - src/args.zig (--help): add XML example with --xml-root / --xml-row
    - docs/sql-pipe.1.scd: add two XML EXAMPLES entries (nested input with
      --xml-root/--xml-row, custom output element names)
    - README.md: expand XML section with RSS-style nested-input example
    
    Closes #144
    vmvarela authored May 10, 2026
    Configuration menu
    Copy the full SHA
    ede09fe View commit details
    Browse the repository at this point in the history
Loading