Skip to content

feat: auto-group and auto-value mode for CSV/JSON tabular data#145

Merged
fahimfaisaal merged 27 commits into
mainfrom
auto-value-mode-ui-changes
Jun 24, 2026
Merged

feat: auto-group and auto-value mode for CSV/JSON tabular data#145
fahimfaisaal merged 27 commits into
mainfrom
auto-value-mode-ui-changes

Conversation

@fahimfaisaal

Copy link
Copy Markdown
Member

Summary

This PR makes CSV and JSON files work out of the box — no -g, -p, or --axes required. When you run vizb bar sales.csv or vizb scatter data.json, vizb now infers how to chart the data automatically:

  1. Auto-group — mixed categorical + numeric tables get a category axis from the most informative non-numeric column
  2. Auto-value — all-numeric tables get the first 2–3 columns mapped to continuous x, y, z value axes

Both modes log their inference to stdout so users can override when the guess is wrong.


Motivation

Previously, tabular data required explicit grouping (-g / -p) or scatter-only --axes for coordinate plots. That made simple workflows like vizb scatter spiral.csv fail or produce empty charts unless users knew the right flags.

The goal: vizb data.csv should just work for the common cases — categorical bar charts and numeric scatter/line/bar coordinate plots.


Auto-grouping

When it runs: CSV/JSON parser, no explicit grouping (-g, -r, custom -p, or --axes), pipeline sets cfg.AutoGroup = true.

Algorithm (pkg/parser/group_spec.goAutoGroupColumns):

  • Scans columns; ignores purely numeric columns
  • Ranks categorical columns by distinct-value count (highest cardinality wins; leftmost tie-break)
  • Picks one column as the x-axis category (-p x)
  • Remaining numeric columns become chart series (stats)

Example:

vizb bar sales.csv -o chart.html
# 🧠 Auto-grouped by column: product

Explicit grouping always wins — any -g, -r, or non-default -p disables auto-group.

3D note: Auto-group sets -p x only. Grouped pseudo-3D (--3d with categorical x+y) still requires explicit -g region,product -p x,y --3d. A warning is logged when --3d is passed without both x and y in the pattern.


Auto-value mode

When it runs: Auto-group finds no categorical columns and the requested chart types include scatter, bar, or line.

Algorithm (AutoValueColumns):

  • Finds the first 2–3 purely numeric columns (in file order)
  • Maps them to value axes: x, y, optional z
  • Each row becomes a raw coordinate point (no stat series)

Example:

vizb scatter spiral.csv -o out.html
# 🧠 Auto-valued by columns (3D pattern x-y-z): x, y, z

--select integration: When --select is set, auto-detect only considers selected columns (FilterHeadersForAutoDetect). Selecting 2 columns forces 2D even if the file has more numeric columns.

Chart eligibility: Auto-value applies to scatter, bar, line only. Pie, heatmap, and radar fall back to flat-series behavior when all columns are numeric.

Auto-3D: When 3 value axes are detected (x, y, z all type: "value"), the pipeline calls autoEnable3DForValueMode to bake --3d into bar/line/scatter configs — continuous WebGL 3D without passing the flag.


Breaking changes

Before After
vizb scatter data.csv --axes x,y vizb scatter data.csv (auto-value) or --select x,y
--axes on scatter subcommand Removed — use auto-value or explicit -g grouping
Hybrid mode (-g + --axes z) Removed — use grouped z column or full auto-value
Auto-group picked 2 columns for 3D (WantsBothXY) Removed — auto-group always picks 1 categorical x

Parser refactor

Extracted shared tabular logic into pkg/parser/group_spec.go:

Helper Purpose
FinalizeGroupConfig Resolve + validate explicit --group config
AutoDetectTabularConfig Run auto-group, else auto-value (early returns, no nested if/else)
FilterHeadersForAutoDetect Scope inference to --select columns

csv.go and json.go now call these helpers instead of duplicating nested control flow.


UI changes

Continuous value-mode 3D rendering for bar, line, and scatter:

  • Shared 3D sizing (ui/src/composables/charts/shared/3d.ts): box extent, camera distance, band fill ratio, bar/symbol sizing for category grids and continuous point clouds
  • Chart composables updated: useBar3DChartOptions, useLine3DChartOptions, useScatter3DChartOptions use shared continuous builders
  • Hybrid dead code removed from transform worker and value-mode paths
  • Settings panel: sort/swap hidden in auto-value 2D (no categorical axes to reorder)
  • Regenerated pkg/template/vizb-ui.gen.go

Examples

New sample datasets for 3D value-mode testing:

  • examples/csv/noise-grid.csv — x,y,z grid (442 rows)
  • examples/csv/noise-surface.csv — x,y,z,value surface (9k+ rows)

Documentation

Updated chart and guide pages:


How to test

# Auto-group: categorical + numeric
vizb bar examples/csv/sales.csv -o /tmp/sales.html

# Auto-value 2D
vizb scatter examples/csv/noise-grid.csv --select x,y -o /tmp/scatter2d.html

# Auto-value 3D (auto-enables --3d)
vizb scatter examples/csv/noise-surface.csv --select x,y,z -o /tmp/scatter3d.html
vizb bar examples/csv/noise-surface.csv --select x,y,z -o /tmp/bar3d.html

# Explicit grouping still works
vizb bar sales.csv -g product -p y -o /tmp/explicit.html

# Unit tests
go test ./...
cd ui && pnpm test

Test coverage

  • pkg/parser/group_spec_test.goAutoGroupColumns, AutoValueColumns, alignment
  • pkg/parser/csv/csv_test.goCSVAutoGroupSuite, CSVAutoValueSuite
  • pkg/parser/json/json_test.goJSONAutoGroupSuite, JSONAutoValueSuite
  • cmd/root_test.goRootAutoGroupSuite end-to-end
  • cmd/charts/scatter/scatter_test.go — auto-value CLI integration
  • ui/src/composables/charts/shared/3d.test.ts — 3D sizing helpers
  • ui/src/workers/transform.worker.test.ts — value-mode transform paths

Commit history (feature-relevant)

b38d45f feat: auto-value-mode for csv/json parsers + remove --axes
1f6173f feat: restrict auto-value to scatter/bar/line (Go) + delete hybrid dead code (UI)
64e1739 feat(parser,ui): auto-value 3D rendering and flatten tabular parser flow

Lead with CSV/JSON across landing, getting-started, and features pages.
Reframe the parsers guide as 'Supported Inputs' with CSV/JSON first and
benchmarks as a supported category. Update the statistics panel docs to
match the current implementation (33 metrics across 7 groups, 4
correlation methods with axis selector, virtualized table, per-z rows
for 3D charts). Refresh the how-it-works internals page to reflect the
current source tree, data shapes, and detection order. Remove the
outdated auto-detection page and clean up its cross-references. Trim
and tighten prose throughout while preserving every example and table.
The Linux and Windows tabs of the landing-page quick example used
trailing comments (`# ...`) on each line that ended with a line-
continuation character (\` for bash/zsh, backtick for PowerShell).
The shell parser consumes the continuation as part of the comment, so
each example broke into multiple top-level commands — vizb got an
empty-string file arg and the rest became `command not found` errors.

Move the descriptions to a single comment block at the top of each tab
so the continuations only ever sit at the end of real command lines.
Add a short note inside each block explaining the gotcha so future
edits don't regress it.
Keep the bash/PowerShell command lines as bare commands — no inline
comments that could collide with the line-continuation characters
(` for bash/zsh, backtick for PowerShell). Describe the flags once
in a bullet list above the OS tabs, and let each tab show only the
shell-specific command. The Linux/macOS bash tab remains the first
and default-selected tab.
The example is more than a generic '3D interactive chart' — the
z/Month/x/Date pattern with `vizb bar` produces a stacked 3D bar
chart where each year is a layer and each day is a column. That
matches the 'GitHub contribution skyline' framing, so say so
explicitly in the intro and hint at the year/day structure.
Rewrite the README intro around CSV/JSON tables with benchmark support
as a secondary capability. Fix 5D→4D framing, rename assests/ to assets/,
and streamline doc links. Add a troubleshooting page, collapse advanced
grouping examples, and ship contributor onboarding (CONTRIBUTING, CoC,
SECURITY, issue templates).
- Remove deprecated scale input (BREAKING; use chart input instead)
- Add select, json-path, stat, chart, and enable-3d inputs
- Update cmd, group, charts, parser, tag-axis, data-url descriptions
- Deprecate sort and show-labels inputs in action descriptions
- Add vizb-binary input to install a pre-built binary, skipping release download
- Merge install/download step with bash-level vizb-binary guard
- Update act tests to build and use local vizb binary
- Ensure act test tasks depend on build:cli
- Add warnDeprecatedRootFlags() in cmd/root.go; called at start of runBenchmark
- Print stderr warning recommending --chart equivalent
- Add resetChanged() helper in cmd/testing.go to clear pflag Changed state
- Add tests for sort/show-labels deprecation warnings and no-warning cases
- Update TestRunBenchmarkGlobalSortApplied to capture stderr
…ions

- Rewrite GitHub Action inputs table: remove scale, add select/json-path/stat/chart/enable-3d/vizb-binary, mark sort/show-labels deprecated
- Add examples for CSV select, JSON path, chart overrides, stats, 3D, and testing local builds
- Update data-url Aside and description to reflect vizb JSON scope
- Update root command flags table: remove scale, deprecate sort/show-labels, update log-scale example
- Add [Unreleased] changelog entry with additions, removals, and deprecations
- AutoValueColumns helper picks first 2-3 all-numeric cols as value axes
- CSV/JSON parsers integrate auto-value when auto-group fails (all-numeric)
- deriveAxesFromData detects value mode (empty Stats) -> Axis.Type:value
- --axes flag removed from scatter cmd; hybrid mode deleted
- IsHybridMode/HybridAxes/ParseAxesFlag deleted
- parseCSVHybridMode/parseJSONHybridMode deleted
- Swap validation moved from scatter Run to pipeline after assembly
- SwapString() added to ChartConfig interface + all 6 configs
- Full test suite green (27 packages)
…ad code (UI)

Part A (Go):
- Add ChartTypes []string field to parser.Config
- Populate ChartTypes from chart configs in pipeline
- Add AutoValueEligible helper (scatter/bar/line only)
- Gate auto-value-mode in csv/json parsers with AutoValueEligible
- Tests: unit tests for AutoValueEligible + integration tests

Part B (UI):
- Delete isHybridMode, buildHybridScatterChart, hybridZStatType
- Remove hybrid mode detection/state/branches from worker
- Remove __hybrid_mode__ skeleton slot from pipeline
- Clean up imports, comments, tests
- 322/322 UI tests pass, all Go tests pass
Auto-value mode (all-numeric CSV/JSON):
- Infer first 2–3 numeric columns as x/y[/z] value axes without --axes
- Respect --select when narrowing which columns participate in auto-detect
- Auto-enable --3d on bar/line/scatter when three value axes are present
- Add noise-grid and noise-surface example datasets for 3D value charts

Parser simplification:
- Extract FinalizeGroupConfig and AutoDetectTabularConfig into group_spec.go
- Replace nested if/else in csv/json parsers with early-return helpers
- Drop WantsBothXY: auto-group always picks a single categorical x column
- Flatten chart-column resolution (default then override) and nil-safe label maps

UI 3D improvements:
- Centralize 3D box sizing, band fill, and camera distance in shared/3d.ts
- Add continuous value-mode 3D builders for bar3D, line3D, and scatter3D
- Remove hybrid mode dead code; unify value-mode line/scatter symbol defaults
- Regenerate embedded UI template

Docs:
- Document auto-value mode; remove --axes and hybrid scatter references
- Update bar, line, scatter, 3d, grouping, and UI guides accordingly
@codecov-commenter

codecov-commenter commented Jun 23, 2026

Copy link
Copy Markdown

Parse optional 4th numeric column as metric in CSV/JSON parsers and
append a metric axis in the CLI pipeline. Auto-enable 3D scatter and
visualMap when metric data is present.

Apply visualMap symbolSize and colorAlpha only when series points carry
a metric dimension; preserve category-grouped 3D behavior otherwise.
Consolidate 4-tuple transforms and pipeline flag helpers.

Update deploy example workflows, docs, and add noise-grid CSV example.
Add targeted Go tests for AutoDetectTabularConfig orchestration,
pipeline auto-3D/metric paths, csv/json value-mode fatal branches,
SwapString helpers, and metric axis swap identity.
Rename ci.yml to cli.yml with lint → format → test → build gates
for Go (golangci-lint, gofmt, tests + Codecov, binary build). Add
ui.yml with the same pipeline for Vue (typecheck, prettier, vitest,
embed build + vizb-ui.gen.go sync). Fix pre-existing errcheck lint
findings so the new lint layer passes on main.
Run gofmt inside embed-ui after codegen so task build:ui and CI compare
formatted output directly. Drop the redundant gofmt step from build:ui
and normalize both sides in the UI workflow gen.go sync check.
Run UI workflow on pushes to any branch (not only main) when ui/,
vizb-ui.gen.go, or ui.yml change. Omit Taskfile.yml from path filters.
readdir order differs across filesystems, so VizbChunks entry order
changed between local builds and ubuntu-slim CI even when chunk content
was identical. Sort chunk files and emitted map keys/refs in embed
codegen, then regenerate vizb-ui.gen.go.
Node zlib can gzip identical JS to different base64 across versions,
so byte-diffing vizb-ui.gen.go failed on ubuntu-slim even when chunk
payloads matched. Compare gunzipped chunk sources plus import graph,
chart roots, entry key, and HTML template instead. Keep sorted map
emission for stable formatting.
@fahimfaisaal
fahimfaisaal merged commit 2853fc2 into main Jun 24, 2026
12 checks passed
@fahimfaisaal
fahimfaisaal deleted the auto-value-mode-ui-changes branch June 24, 2026 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants