feat: add optional per-issue JSON metadata field (SQLite + Dolt) by turian · Pull Request #1407 · steveyegge/beads

turian · 2026-01-31T03:42:23Z

Summary

This PR adds an optional metadata field to types.Issue to support structured, extensible attachments (tool annotations, file lists, integration payloads, etc.) without requiring new first-class columns for every use case.

The field is:

stored in SQLite (TEXT NOT NULL DEFAULT '{}') and Dolt (JSON DEFAULT (JSON_OBJECT()))
validated as well-formed JSON on create/update/import
included in issue content hashing (so metadata-only changes are treated as real content changes)
covered by new SQLite round-trip tests (create/get/search/update)

Motivation / Use Cases

Beads already uses JSON blobs in a few places (e.g. dependency metadata, event payloads, JSON-encoded arrays). This PR extends that pattern to issues themselves.

Example use cases:

File lists / affected files: {"files":["foo.go","bar.go"]} (explicit filenames/paths; no globs)
Tool annotations: store linter/formatter output summaries, "last checked" timestamps, or tool configuration snapshots
External system bridging: store adapter-specific IDs or state (e.g. SARIF run IDs, CI job URLs, Jira/Linear metadata) without expanding the core schema
Agent / automation state: store structured "why this was created" context, provenance, or small machine-readable hints for downstream automation
Diagnostics payloads: lightweight structured data attached to an issue (related to SARIF-style workflows)

Implementation Details

internal/types/types.go
- Adds Metadata json.RawMessage \json:"metadata,omitempty"``
- Validates metadata is well-formed JSON in ValidateWithCustom and ValidateForImport
- Includes metadata in ComputeContentHash()
SQLite
- Adds issues.metadata column via new migration 041_metadata_column.go
- Updates all relevant SELECT/scan paths to include metadata (GetIssue/SearchIssues/ready/labels/transaction scan helpers)
- Allows updates via UpdateIssue (allowedUpdateFields["metadata"]=true)
- Adds internal/storage/sqlite/metadata_test.go to verify create/get/search/update and nil behavior when unset
Dolt
- Adds issues.metadata JSON column in schema
- Ensures inserts always write valid JSON using jsonMetadata() helper (empty → "{}")
- Scans metadata back into types.Issue
- Allows updates via isAllowedUpdateField("metadata")

Compatibility / Why this shouldn't break existing users

This is an additive, backwards-compatible change for typical Beads usage:

Existing JSONL files remain valid: metadata is optional (omitempty). Older JSONL without metadata imports as before.
Existing SQLite databases migrate automatically: the new column is added via an idempotent migration with a safe default ('{}').
Existing workflows are unchanged unless users opt into setting metadata.

Potential edge cases:

Strict JSON consumers: external tools that parse issue JSON with "reject unknown fields" semantics may need to tolerate the new metadata field.
Existing Dolt databases: if a Dolt DB was created before this change, it may require an ALTER TABLE to add the new column (Dolt schema creation is CREATE TABLE IF NOT EXISTS, which does not modify existing tables). New databases are unaffected.

Related Issues / Context

Closes feat: add optional JSON metadata field to issues #1406
Related: Add support for importing and tracking SARIFs in beads #1377 (structured diagnostic/SARIF-style data)

Follow-ups (proposed)

CLI/RPC support for setting metadata (e.g. bd update --metadata @file.json or bd annotate --metadata ...)
Optional schema enforcement (config-driven; either Go-struct validation per issue type or JSON Schema files)
Conventions for common keys (documented, not enforced), e.g.:
- metadata.files: explicit filenames/paths only (no globs)
- metadata.tool: tool name/version
- metadata.diagnostics: structured findings summary
Dolt migration path for existing databases (if desired): add an automatic ALTER TABLE step during init/open when column missing.

🤖 Generated with Claude Code

Add an optional `metadata` field to Issue for storing arbitrary JSON data. This enables extension points for tool annotations, file lists, and external system bridging without expanding the core schema. Changes: - Add `Metadata json.RawMessage` field to types.Issue - Add metadata column to SQLite and Dolt schemas - Add migration 041_metadata_column for existing databases - Wire through insert/update/get/search in both backends - Validate metadata is well-formed JSON on create/update - Include metadata in content hash computation The field uses json.RawMessage to preserve exact JSON and supports omitempty for JSONL export compactness. Closes steveyegge#1406 Co-Authored-By: Claude Opus 4.5 <[email protected]>

Include metadata column in: - SearchIssues SELECT statement (queries.go) - scanIssues function in dependencies.go - transaction.go GetIssue and SearchIssues SELECTs - scanIssueRow helper function Without this, bd list/ready and other search operations would return issues with empty Metadata fields even when stored. Co-Authored-By: Claude Opus 4.5 <[email protected]>

- Add metadata to GetIssuesByLabel in labels.go - Add metadata to GetReadyWork and GetNewlyUnblockedByClose in ready.go - Fix TestMigrateContentHashColumn test to include metadata column This fixes test failures from the initial metadata implementation where some SELECT statements were missing the new metadata column. Co-Authored-By: Claude Opus 4.5 <[email protected]>

- Add "metadata" to SQLite allowedUpdateFields (matches Dolt) - Add TestIssueMetadataRoundTrip test verifying: - Create issue with metadata -> GetIssue returns it - SearchIssues returns metadata - UpdateIssue can modify metadata - Issues without metadata have nil Metadata field This ensures metadata can be updated via bd update and proves the feature works across key code paths (GH#1406). Co-Authored-By: Claude Opus 4.5 <[email protected]>

Dolt's JSON column requires valid JSON, so empty strings cause "Invalid JSON text" errors. Added jsonMetadata() helper that returns "{}" when metadata is nil or empty. This fixes TestBootstrapFromJSONL and related Dolt tests. Co-Authored-By: Claude Opus 4.5 <[email protected]>

Replace string comparison with jsonEqual() helper that unmarshals and uses reflect.DeepEqual. This prevents test brittleness if JSON gets normalized (whitespace, key order) in the future. Co-Authored-By: Claude Opus 4.5 <[email protected]>

turian · 2026-01-31T05:35:51Z

Proposed Follow-ups

Tracking items for post-merge work:

1. CLI support for setting metadata

Add bd update <id> --metadata '{"key":"value"}' or bd update <id> --metadata @file.json to enable setting metadata from the command line. Currently metadata can only be set programmatically.

2. Dolt migration path for existing databases

Existing Dolt databases won't automatically get the new metadata column (schema uses CREATE TABLE IF NOT EXISTS). Options:

Add automatic ALTER TABLE detection during init/open when column is missing
Document manual migration: ALTER TABLE issues ADD COLUMN metadata JSON DEFAULT (JSON_OBJECT())

3. Conventions for common metadata keys (documentation)

Document recommended patterns without enforcing them:

metadata.files - explicit filenames/paths only (no globs)
metadata.tool - tool name/version that created the issue
metadata.diagnostics - structured findings (SARIF-style)
metadata.external_ids - mapping to external systems (Linear, Jira, GitHub)

4. Optional schema enforcement (future)

Config-driven validation beyond json.Valid():

Per-type schemas (e.g. "files must be array of strings")
JSON Schema support for custom validators

steveyegge

Clean additive feature. Metadata field works well with Dolt's native JSON type and is compatible with the Dolt-only storage direction. Good test coverage, proper validation, correct content hash inclusion. LGTM.

…veyegge#1407) * feat: add optional JSON metadata field to issues Add an optional `metadata` field to Issue for storing arbitrary JSON data. This enables extension points for tool annotations, file lists, and external system bridging without expanding the core schema. Changes: - Add `Metadata json.RawMessage` field to types.Issue - Add metadata column to SQLite and Dolt schemas - Add migration 041_metadata_column for existing databases - Wire through insert/update/get/search in both backends - Validate metadata is well-formed JSON on create/update - Include metadata in content hash computation The field uses json.RawMessage to preserve exact JSON and supports omitempty for JSONL export compactness. Closes steveyegge#1406 Co-Authored-By: Claude Opus 4.5 <[email protected]> * fix: add metadata to SQLite SearchIssues and scanIssues Include metadata column in: - SearchIssues SELECT statement (queries.go) - scanIssues function in dependencies.go - transaction.go GetIssue and SearchIssues SELECTs - scanIssueRow helper function Without this, bd list/ready and other search operations would return issues with empty Metadata fields even when stored. Co-Authored-By: Claude Opus 4.5 <[email protected]> * fix: add metadata column to remaining SELECT statements - Add metadata to GetIssuesByLabel in labels.go - Add metadata to GetReadyWork and GetNewlyUnblockedByClose in ready.go - Fix TestMigrateContentHashColumn test to include metadata column This fixes test failures from the initial metadata implementation where some SELECT statements were missing the new metadata column. Co-Authored-By: Claude Opus 4.5 <[email protected]> * feat: enable metadata updates and add round-trip tests - Add "metadata" to SQLite allowedUpdateFields (matches Dolt) - Add TestIssueMetadataRoundTrip test verifying: - Create issue with metadata -> GetIssue returns it - SearchIssues returns metadata - UpdateIssue can modify metadata - Issues without metadata have nil Metadata field This ensures metadata can be updated via bd update and proves the feature works across key code paths (GH#1406). Co-Authored-By: Claude Opus 4.5 <[email protected]> * fix(dolt): handle empty metadata for JSON column type Dolt's JSON column requires valid JSON, so empty strings cause "Invalid JSON text" errors. Added jsonMetadata() helper that returns "{}" when metadata is nil or empty. This fixes TestBootstrapFromJSONL and related Dolt tests. Co-Authored-By: Claude Opus 4.5 <[email protected]> * test: use structural JSON comparison in metadata tests Replace string comparison with jsonEqual() helper that unmarshals and uses reflect.DeepEqual. This prevents test brittleness if JSON gets normalized (whitespace, key order) in the future. Co-Authored-By: Claude Opus 4.5 <[email protected]> --------- Co-authored-by: Claude Opus 4.5 <[email protected]> (cherry picked from commit 92fd124)

Add support for setting custom JSON metadata on issues via the bd update command: - Inline JSON: bd update <id> --metadata '{"key":"value"}' - File reference: bd update <id> --metadata @file.json Implementation details: - Add Metadata field to UpdateArgs in RPC protocol - Validate JSON with json.Valid() before passing to storage - Parse @file.json syntax to read metadata from files - Add tests for both inline and file-based metadata input The metadata field is already included in bd show --json output via the Issue struct's Metadata field (from PR steveyegge#1407). Closes steveyegge#1413 Co-Authored-By: Claude Opus 4.5 <[email protected]>

turian and others added 6 commits January 30, 2026 19:42

turian changed the title ~~feat: add optional JSON metadata field to issues~~ feat: add optional per-issue JSON metadata field (SQLite + Dolt) Jan 31, 2026

steveyegge approved these changes Jan 31, 2026

View reviewed changes

steveyegge merged commit 92fd124 into steveyegge:main Jan 31, 2026

steveyegge mentioned this pull request Feb 14, 2026

Feature: Plugin-based issue tracker integrations (Jira, Linear, Azure DevOps, etc.) #1150

Open

turian mentioned this pull request Feb 20, 2026

feat(metadata): human-readable visibility in bd show and bd list --long #1905

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add optional per-issue JSON metadata field (SQLite + Dolt)#1407

feat: add optional per-issue JSON metadata field (SQLite + Dolt)#1407
steveyegge merged 6 commits intosteveyegge:mainfrom
turian:feat/issue-metadata

turian commented Jan 31, 2026 •

edited

Loading

Uh oh!

turian commented Jan 31, 2026

Uh oh!

steveyegge left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

turian commented Jan 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation / Use Cases

Implementation Details

Compatibility / Why this shouldn't break existing users

Related Issues / Context

Follow-ups (proposed)

Uh oh!

turian commented Jan 31, 2026

Proposed Follow-ups

1. CLI support for setting metadata

2. Dolt migration path for existing databases

3. Conventions for common metadata keys (documentation)

4. Optional schema enforcement (future)

Uh oh!

steveyegge left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

turian commented Jan 31, 2026 •

edited

Loading