Skip to content

Commit 316bfc6

Browse files
pelletierclaude
pelletier
and
claude
authored
Support Unmarshaler interface for tables and array tables (#1027)
Fixes #873 Extend the unstable.Unmarshaler interface support to work with tables and array tables, not just single values. When a type implementing unstable.Unmarshaler is the target of a table (e.g., [table] or [[array]]), the UnmarshalTOML method receives a synthetic InlineTable node containing all the key-value pairs belonging to that table. Key changes: - Add handleKeyValuesUnmarshaler to collect and process table content - Add copyExpressionNodes to deep-copy AST nodes for synthetic tables - Add helper functions in unstable/ast.go for node manipulation - Update documentation for EnableUnmarshalerInterface - Add comprehensive tests for table and array table unmarshaling * Implement bytes-based Unmarshaler interface for tables and arrays (#873) This change brings back support for the unstable.Unmarshaler interface for tables and array tables, addressing issue #873. Key changes: - Changed UnmarshalTOML signature from (*Node) to ([]byte) to provide raw TOML bytes instead of AST nodes - Added RawMessage type (similar to json.RawMessage) for capturing raw TOML bytes for later processing - Updated handleKeyValuesUnmarshaler to reconstruct key-value lines from the parsed keys and raw value bytes - Added support for slice types implementing Unmarshaler (e.g., RawMessage) - Removed unused AST helper functions from unstable/ast.go The bytes-based interface allows users to: - Get raw TOML bytes for custom parsing - Delay TOML decoding using RawMessage - Implement custom unmarshaling logic for complex types Tests added for: - Table unmarshaler with various scenarios - Array table unmarshaler - Split tables (same parent defined in multiple places) - RawMessage usage - Nested tables and mixed regular fields * Fix lint issues and improve test coverage for Unmarshaler interface - Apply De Morgan's law in keyNeedsQuoting to satisfy staticcheck QF1001 - Remove unused splitTableUnmarshaler type from test - Fix unused parameter lint warning in errorUnmarshaler873 - Add test for quoted keys that need special handling - Add test for error propagation from UnmarshalTOML - Update customTable873 parser to handle quoted keys properly Coverage improved: - handleKeyValuesUnmarshaler: 80.0% -> 93.3% - keyNeedsQuoting: 66.7% -> 83.3% - Overall main package: 97.2% -> 97.5% * Add test for dotted keys to improve coverage Add TestIssue873_DottedKeys to test dotted key handling (e.g., sub.key = value) in the Unmarshaler interface. This improves coverage for handleKeyValuesUnmarshaler from 93.3% to 96.7%. * Add double pointer test to achieve 100% coverage for handleKeyValues Add TestIssue873_DoublePointerUnmarshaler to test pointer-to-pointer to Unmarshaler types. This covers the pointer dereferencing loop in handleKeyValues, bringing its coverage from 88% to 100%. Total coverage: 97.4% * Add Example tests and fix raw value extraction for boolean types Add two godoc Example tests: - ExampleDecoder_EnableUnmarshalerInterface_dynamicConfig: shows dynamic unmarshaling based on a type field - ExampleDecoder_EnableUnmarshalerInterface_rawMessage: demonstrates RawMessage usage for deferred parsing Fix handleKeyValuesUnmarshaler to handle values where Raw.Length == 0 (like boolean types) by using value.Data as fallback. * Preserve original formatting in Unmarshaler by using raw byte ranges Instead of reconstructing key-value lines from parsed components, now uses the original raw bytes from the document. This preserves: - Whitespace around '=' (e.g., "key = value") - String quoting style (basic vs literal) - Number formats (hex, octal, binary) - Inline table formatting Changes: - Add Raw range tracking to KeyValue expressions in parseKeyval - Update handleKeyValuesUnmarshaler to use expr.Raw directly - Remove keyNeedsQuoting helper (no longer needed) - Add TestIssue873_FormattingPreservation test - Update expected output in ExampleParser_comments * Prevent test matrix from canceling on first failure Add fail-fast: false to the test workflow strategy so that all OS/Go version combinations continue running even if one fails. This provides better visibility into which specific combinations have issues. --------- Co-authored-by: Claude <[email protected]>
1 parent 2edc61f commit 316bfc6

6 files changed

Lines changed: 522 additions & 30 deletions

File tree

‎.github/workflows/workflow.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ on:
1010
jobs:
1111
build:
1212
strategy:
13+
fail-fast: false
1314
matrix:
1415
os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest', 'macos-14' ]
1516
go: [ '1.24', '1.25' ]

‎unmarshaler.go‎

Lines changed: 85 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -56,13 +56,18 @@ func (d *Decoder) DisallowUnknownFields() *Decoder {
5656

5757
// EnableUnmarshalerInterface allows to enable unmarshaler interface.
5858
//
59-
// With this feature enabled, types implementing the unstable/Unmarshaler
59+
// With this feature enabled, types implementing the unstable.Unmarshaler
6060
// interface can be decoded from any structure of the document. It allows types
6161
// that don't have a straightforward TOML representation to provide their own
6262
// decoding logic.
6363
//
64-
// Currently, types can only decode from a single value. Tables and array tables
65-
// are not supported.
64+
// The UnmarshalTOML method receives raw TOML bytes:
65+
// - For single values: the raw value bytes (e.g., `"hello"` for a string)
66+
// - For tables: all key-value lines belonging to that table
67+
// - For inline tables/arrays: the raw bytes of the inline structure
68+
//
69+
// The unstable.RawMessage type can be used to capture raw TOML bytes for
70+
// later processing, similar to json.RawMessage.
6671
//
6772
// *Unstable:* This method does not follow the compatibility guarantees of
6873
// semver. It can be changed or removed without a new major version being
@@ -599,18 +604,28 @@ func (d *decoder) handleArrayTablePart(key unstable.Iterator, v reflect.Value) (
599604
// cannot handle it.
600605
func (d *decoder) handleTable(key unstable.Iterator, v reflect.Value) (reflect.Value, error) {
601606
if v.Kind() == reflect.Slice {
602-
if v.Len() == 0 {
603-
return reflect.Value{}, unstable.NewParserError(key.Node().Data, "cannot store a table in a slice")
604-
}
605-
elem := v.Index(v.Len() - 1)
606-
x, err := d.handleTable(key, elem)
607-
if err != nil {
608-
return reflect.Value{}, err
607+
// For non-empty slices, work with the last element
608+
if v.Len() > 0 {
609+
elem := v.Index(v.Len() - 1)
610+
x, err := d.handleTable(key, elem)
611+
if err != nil {
612+
return reflect.Value{}, err
613+
}
614+
if x.IsValid() {
615+
elem.Set(x)
616+
}
617+
return reflect.Value{}, nil
609618
}
610-
if x.IsValid() {
611-
elem.Set(x)
619+
// Empty slice - check if it implements Unmarshaler (e.g., RawMessage)
620+
// and we're at the end of the key path
621+
if d.unmarshalerInterface && !key.Next() {
622+
if v.CanAddr() && v.Addr().CanInterface() {
623+
if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok {
624+
return d.handleKeyValuesUnmarshaler(outi)
625+
}
626+
}
612627
}
613-
return reflect.Value{}, nil
628+
return reflect.Value{}, unstable.NewParserError(key.Node().Data, "cannot store a table in a slice")
614629
}
615630
if key.Next() {
616631
// Still scoping the key
@@ -624,6 +639,24 @@ func (d *decoder) handleTable(key unstable.Iterator, v reflect.Value) (reflect.V
624639
// Handle root expressions until the end of the document or the next
625640
// non-key-value.
626641
func (d *decoder) handleKeyValues(v reflect.Value) (reflect.Value, error) {
642+
// Check if target implements Unmarshaler before processing key-values.
643+
// This allows types to handle entire tables themselves.
644+
if d.unmarshalerInterface {
645+
vv := v
646+
for vv.Kind() == reflect.Ptr {
647+
if vv.IsNil() {
648+
vv.Set(reflect.New(vv.Type().Elem()))
649+
}
650+
vv = vv.Elem()
651+
}
652+
if vv.CanAddr() && vv.Addr().CanInterface() {
653+
if outi, ok := vv.Addr().Interface().(unstable.Unmarshaler); ok {
654+
// Collect all key-value expressions for this table
655+
return d.handleKeyValuesUnmarshaler(outi)
656+
}
657+
}
658+
}
659+
627660
var rv reflect.Value
628661
for d.nextExpr() {
629662
expr := d.expr()
@@ -653,6 +686,41 @@ func (d *decoder) handleKeyValues(v reflect.Value) (reflect.Value, error) {
653686
return rv, nil
654687
}
655688

689+
// handleKeyValuesUnmarshaler collects all key-value expressions for a table
690+
// and passes them to the Unmarshaler as raw TOML bytes.
691+
func (d *decoder) handleKeyValuesUnmarshaler(u unstable.Unmarshaler) (reflect.Value, error) {
692+
// Collect raw bytes from all key-value expressions for this table.
693+
// We use the Raw field on each KeyValue expression to preserve the
694+
// original formatting (whitespace, quoting style, etc.) from the document.
695+
var buf []byte
696+
697+
for d.nextExpr() {
698+
expr := d.expr()
699+
if expr.Kind != unstable.KeyValue {
700+
d.stashExpr()
701+
break
702+
}
703+
704+
_, err := d.seen.CheckExpression(expr)
705+
if err != nil {
706+
return reflect.Value{}, err
707+
}
708+
709+
// Use the raw bytes from the original document to preserve formatting
710+
if expr.Raw.Length > 0 {
711+
raw := d.p.Raw(expr.Raw)
712+
buf = append(buf, raw...)
713+
}
714+
buf = append(buf, '\n')
715+
}
716+
717+
if err := u.UnmarshalTOML(buf); err != nil {
718+
return reflect.Value{}, err
719+
}
720+
721+
return reflect.Value{}, nil
722+
}
723+
656724
type (
657725
handlerFn func(key unstable.Iterator, v reflect.Value) (reflect.Value, error)
658726
valueMakerFn func() reflect.Value
@@ -697,7 +765,8 @@ func (d *decoder) handleValue(value *unstable.Node, v reflect.Value) error {
697765
if d.unmarshalerInterface {
698766
if v.CanAddr() && v.Addr().CanInterface() {
699767
if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok {
700-
return outi.UnmarshalTOML(value)
768+
// Pass raw bytes from the original document
769+
return outi.UnmarshalTOML(d.p.Raw(value.Raw))
701770
}
702771
}
703772
}
@@ -1201,7 +1270,8 @@ func (d *decoder) handleKeyValuePart(key unstable.Iterator, value *unstable.Node
12011270
if d.unmarshalerInterface {
12021271
if v.CanAddr() && v.Addr().CanInterface() {
12031272
if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok {
1204-
return reflect.Value{}, outi.UnmarshalTOML(value)
1273+
// Pass raw bytes from the original document
1274+
return reflect.Value{}, outi.UnmarshalTOML(d.p.Raw(value.Raw))
12051275
}
12061276
}
12071277
}

0 commit comments

Comments
 (0)