feat: auto-group and auto-value mode for CSV/JSON tabular data#145
Merged
Conversation
Lead with CSV/JSON across landing, getting-started, and features pages. Reframe the parsers guide as 'Supported Inputs' with CSV/JSON first and benchmarks as a supported category. Update the statistics panel docs to match the current implementation (33 metrics across 7 groups, 4 correlation methods with axis selector, virtualized table, per-z rows for 3D charts). Refresh the how-it-works internals page to reflect the current source tree, data shapes, and detection order. Remove the outdated auto-detection page and clean up its cross-references. Trim and tighten prose throughout while preserving every example and table.
The Linux and Windows tabs of the landing-page quick example used trailing comments (`# ...`) on each line that ended with a line- continuation character (\` for bash/zsh, backtick for PowerShell). The shell parser consumes the continuation as part of the comment, so each example broke into multiple top-level commands — vizb got an empty-string file arg and the rest became `command not found` errors. Move the descriptions to a single comment block at the top of each tab so the continuations only ever sit at the end of real command lines. Add a short note inside each block explaining the gotcha so future edits don't regress it.
Keep the bash/PowerShell command lines as bare commands — no inline comments that could collide with the line-continuation characters (` for bash/zsh, backtick for PowerShell). Describe the flags once in a bullet list above the OS tabs, and let each tab show only the shell-specific command. The Linux/macOS bash tab remains the first and default-selected tab.
The example is more than a generic '3D interactive chart' — the z/Month/x/Date pattern with `vizb bar` produces a stacked 3D bar chart where each year is a layer and each day is a column. That matches the 'GitHub contribution skyline' framing, so say so explicitly in the intro and hint at the year/day structure.
Rewrite the README intro around CSV/JSON tables with benchmark support as a secondary capability. Fix 5D→4D framing, rename assests/ to assets/, and streamline doc links. Add a troubleshooting page, collapse advanced grouping examples, and ship contributor onboarding (CONTRIBUTING, CoC, SECURITY, issue templates).
- Remove deprecated scale input (BREAKING; use chart input instead) - Add select, json-path, stat, chart, and enable-3d inputs - Update cmd, group, charts, parser, tag-axis, data-url descriptions - Deprecate sort and show-labels inputs in action descriptions - Add vizb-binary input to install a pre-built binary, skipping release download - Merge install/download step with bash-level vizb-binary guard - Update act tests to build and use local vizb binary - Ensure act test tasks depend on build:cli
- Add warnDeprecatedRootFlags() in cmd/root.go; called at start of runBenchmark - Print stderr warning recommending --chart equivalent - Add resetChanged() helper in cmd/testing.go to clear pflag Changed state - Add tests for sort/show-labels deprecation warnings and no-warning cases - Update TestRunBenchmarkGlobalSortApplied to capture stderr
…ions - Rewrite GitHub Action inputs table: remove scale, add select/json-path/stat/chart/enable-3d/vizb-binary, mark sort/show-labels deprecated - Add examples for CSV select, JSON path, chart overrides, stats, 3D, and testing local builds - Update data-url Aside and description to reflect vizb JSON scope - Update root command flags table: remove scale, deprecate sort/show-labels, update log-scale example - Add [Unreleased] changelog entry with additions, removals, and deprecations
- AutoValueColumns helper picks first 2-3 all-numeric cols as value axes - CSV/JSON parsers integrate auto-value when auto-group fails (all-numeric) - deriveAxesFromData detects value mode (empty Stats) -> Axis.Type:value - --axes flag removed from scatter cmd; hybrid mode deleted - IsHybridMode/HybridAxes/ParseAxesFlag deleted - parseCSVHybridMode/parseJSONHybridMode deleted - Swap validation moved from scatter Run to pipeline after assembly - SwapString() added to ChartConfig interface + all 6 configs - Full test suite green (27 packages)
…ad code (UI) Part A (Go): - Add ChartTypes []string field to parser.Config - Populate ChartTypes from chart configs in pipeline - Add AutoValueEligible helper (scatter/bar/line only) - Gate auto-value-mode in csv/json parsers with AutoValueEligible - Tests: unit tests for AutoValueEligible + integration tests Part B (UI): - Delete isHybridMode, buildHybridScatterChart, hybridZStatType - Remove hybrid mode detection/state/branches from worker - Remove __hybrid_mode__ skeleton slot from pipeline - Clean up imports, comments, tests - 322/322 UI tests pass, all Go tests pass
Auto-value mode (all-numeric CSV/JSON): - Infer first 2–3 numeric columns as x/y[/z] value axes without --axes - Respect --select when narrowing which columns participate in auto-detect - Auto-enable --3d on bar/line/scatter when three value axes are present - Add noise-grid and noise-surface example datasets for 3D value charts Parser simplification: - Extract FinalizeGroupConfig and AutoDetectTabularConfig into group_spec.go - Replace nested if/else in csv/json parsers with early-return helpers - Drop WantsBothXY: auto-group always picks a single categorical x column - Flatten chart-column resolution (default then override) and nil-safe label maps UI 3D improvements: - Centralize 3D box sizing, band fill, and camera distance in shared/3d.ts - Add continuous value-mode 3D builders for bar3D, line3D, and scatter3D - Remove hybrid mode dead code; unify value-mode line/scatter symbol defaults - Regenerate embedded UI template Docs: - Document auto-value mode; remove --axes and hybrid scatter references - Update bar, line, scatter, 3d, grouping, and UI guides accordingly
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Parse optional 4th numeric column as metric in CSV/JSON parsers and append a metric axis in the CLI pipeline. Auto-enable 3D scatter and visualMap when metric data is present. Apply visualMap symbolSize and colorAlpha only when series points carry a metric dimension; preserve category-grouped 3D behavior otherwise. Consolidate 4-tuple transforms and pipeline flag helpers. Update deploy example workflows, docs, and add noise-grid CSV example.
Add targeted Go tests for AutoDetectTabularConfig orchestration, pipeline auto-3D/metric paths, csv/json value-mode fatal branches, SwapString helpers, and metric axis swap identity.
Rename ci.yml to cli.yml with lint → format → test → build gates for Go (golangci-lint, gofmt, tests + Codecov, binary build). Add ui.yml with the same pipeline for Vue (typecheck, prettier, vitest, embed build + vizb-ui.gen.go sync). Fix pre-existing errcheck lint findings so the new lint layer passes on main.
Run gofmt inside embed-ui after codegen so task build:ui and CI compare formatted output directly. Drop the redundant gofmt step from build:ui and normalize both sides in the UI workflow gen.go sync check.
Run UI workflow on pushes to any branch (not only main) when ui/, vizb-ui.gen.go, or ui.yml change. Omit Taskfile.yml from path filters.
readdir order differs across filesystems, so VizbChunks entry order changed between local builds and ubuntu-slim CI even when chunk content was identical. Sort chunk files and emitted map keys/refs in embed codegen, then regenerate vizb-ui.gen.go.
Node zlib can gzip identical JS to different base64 across versions, so byte-diffing vizb-ui.gen.go failed on ubuntu-slim even when chunk payloads matched. Compare gunzipped chunk sources plus import graph, chart roots, entry key, and HTML template instead. Keep sorted map emission for stable formatting.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR makes CSV and JSON files work out of the box — no
-g,-p, or--axesrequired. When you runvizb bar sales.csvorvizb scatter data.json, vizb now infers how to chart the data automatically:x,y,zvalue axesBoth modes log their inference to stdout so users can override when the guess is wrong.
Motivation
Previously, tabular data required explicit grouping (
-g/-p) or scatter-only--axesfor coordinate plots. That made simple workflows likevizb scatter spiral.csvfail or produce empty charts unless users knew the right flags.The goal:
vizb data.csvshould just work for the common cases — categorical bar charts and numeric scatter/line/bar coordinate plots.Auto-grouping
When it runs: CSV/JSON parser, no explicit grouping (
-g,-r, custom-p, or--axes), pipeline setscfg.AutoGroup = true.Algorithm (
pkg/parser/group_spec.go→AutoGroupColumns):-p x)Example:
vizb bar sales.csv -o chart.html # 🧠 Auto-grouped by column: productExplicit grouping always wins — any
-g,-r, or non-default-pdisables auto-group.3D note: Auto-group sets
-p xonly. Grouped pseudo-3D (--3dwith categorical x+y) still requires explicit-g region,product -p x,y --3d. A warning is logged when--3dis passed without both x and y in the pattern.Auto-value mode
When it runs: Auto-group finds no categorical columns and the requested chart types include scatter, bar, or line.
Algorithm (
AutoValueColumns):x,y, optionalzExample:
vizb scatter spiral.csv -o out.html # 🧠 Auto-valued by columns (3D pattern x-y-z): x, y, z--selectintegration: When--selectis set, auto-detect only considers selected columns (FilterHeadersForAutoDetect). Selecting 2 columns forces 2D even if the file has more numeric columns.Chart eligibility: Auto-value applies to scatter, bar, line only. Pie, heatmap, and radar fall back to flat-series behavior when all columns are numeric.
Auto-3D: When 3 value axes are detected (
x,y,zalltype: "value"), the pipeline callsautoEnable3DForValueModeto bake--3dinto bar/line/scatter configs — continuous WebGL 3D without passing the flag.Breaking changes
vizb scatter data.csv --axes x,yvizb scatter data.csv(auto-value) or--select x,y--axeson scatter subcommand-ggrouping-g+--axes z)WantsBothXY)Parser refactor
Extracted shared tabular logic into
pkg/parser/group_spec.go:FinalizeGroupConfig--groupconfigAutoDetectTabularConfigFilterHeadersForAutoDetect--selectcolumnscsv.goandjson.gonow call these helpers instead of duplicating nested control flow.UI changes
Continuous value-mode 3D rendering for bar, line, and scatter:
ui/src/composables/charts/shared/3d.ts): box extent, camera distance, band fill ratio, bar/symbol sizing for category grids and continuous point cloudsuseBar3DChartOptions,useLine3DChartOptions,useScatter3DChartOptionsuse shared continuous builderspkg/template/vizb-ui.gen.goExamples
New sample datasets for 3D value-mode testing:
examples/csv/noise-grid.csv— x,y,z grid (442 rows)examples/csv/noise-surface.csv— x,y,z,value surface (9k+ rows)Documentation
Updated chart and guide pages:
--axesand hybrid scatter referencesHow to test
Test coverage
pkg/parser/group_spec_test.go—AutoGroupColumns,AutoValueColumns, alignmentpkg/parser/csv/csv_test.go—CSVAutoGroupSuite,CSVAutoValueSuitepkg/parser/json/json_test.go—JSONAutoGroupSuite,JSONAutoValueSuitecmd/root_test.go—RootAutoGroupSuiteend-to-endcmd/charts/scatter/scatter_test.go— auto-value CLI integrationui/src/composables/charts/shared/3d.test.ts— 3D sizing helpersui/src/workers/transform.worker.test.ts— value-mode transform pathsCommit history (feature-relevant)