initial-default+write-default {} (struct default value) representation issue #16596

Apache Iceberg version

1.11.0 (latest release)

Query engine

None

Please describe the bug 🐞

This issue is similar to #15932

The spec has a very clear table listing behavior for struct defaults, and makes it clear that a struct column can either have null as its default value, or {}, the default values for its fields should be decomposed.

A non-null default is stored by setting initial-default or write-default to an empty struct ({}) that will use field values set from each field's initial-default or write-default, respectively.

But in the Iceberg REST API spec, both initial-default and write-default are modeled as a PrimitiveTypeValue:

        initial-default:
          $ref: "#/components/schemas/PrimitiveTypeValue"
        write-default:
          $ref: "#/components/schemas/PrimitiveTypeValue"

So I don't think we can currently represent {} for an initial-default or write-default

Example request:

{
	"stage-create": true,
	"name": "tbl",
	"schema": {
		"type": "struct",
		"fields": [
			{
				"name": "col",
				"id": 1,
				"type": {
					"type": "struct",
					"fields": [
						{
							"name": "a",
							"id": 2,
							"type": "string",
							"required": false,
							"initial-default": "test"
						},
						{
							"name": "b",
							"id": 3,
							"type": "int",
							"required": false
						}
					]
				},
				"required": false,
				"initial-default": {}
			},
			{
				"name": "col2",
				"id": 4,
				"type": "int",
				"required": false
			}
		],
		"schema-id": 0
	},
	"partition-spec": {
		"spec-id": 0,
		"type": "struct",
		"fields": []
	},
	"write-order": {
		"order-id": 0,
		"fields": []
	},
	"properties": {
		"format-version": "3"
	}
}

Attempting to POST this to the create table endpoint targeting an iceberg-rest-fixture catalog results in a long Internal Error message, of which an excerpt is:

Cannot create expression literal from org.apache.iceberg.data.GenericRecord: Record(null, null)

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time