[Bug] Accept legacy schema JSON without version using Java-compatible defaults #1384

Search before asking

• I searched in the issues and found nothing similar.

Searched issues and PRs for legacy schema, schema version, missing version, and serde. No directly matching issue was found as of November 12, 2026.

Please describe the bug

At paimon-rust main f91fdb8ab034887547a5ee2f26c1979e30250558, TableSchema derives Deserialize and requires a version: i32 field without a missing-field default. The C catalog-free table constructors also deserialize this type directly. A legacy schema JSON without version therefore cannot use those paths.

Java's SchemaSerializer.deserialize accepts a missing version as PAIMON_07_VERSION. It also applies legacy option defaults: bucket 1 when the schema version is at most PAIMON_07_VERSION and bucket is absent, and file format orc when the version is at most PAIMON_08_VERSION and file.format is absent.

Apache Doris currently keeps these historical schemas off its Rust reader path. Its existing Java-backed regression removes version from a real schema file, reopens and reads the table, then evolves the schema and reads the historical data again.

Reproduction for the parser boundary:

1. Take a valid schema JSON with explicit bucket and file.format options.
2. Remove only its version member.
3. Deserialize it as paimon::spec::TableSchema, or pass it to paimon_table_from_schema_json.
4. Compare with Java's TableSchema.fromJson on the same JSON.

Expected: missing-version metadata receives the Java-compatible legacy interpretation. Explicit versions retain their meaning. This request does not ask to accept arbitrarily malformed metadata.

The evidence above is source inspection and an existing Doris test; no new upstream test was executed for this report.

Solution

Support the legacy missing-version interpretation in the shared schema-loading path. Check existing normalization before adding defaults; a bare #[serde(default)] would produce 0 rather than Java's legacy version.

Compatibility coverage should include explicit versions, missing version, malformed version values, current and historical schemas, and missing legacy bucket/file.format options. A parser-only success should not be treated as proof that historical table reads use the correct storage interpretation.

References:

• Rust TableSchema
• Java SchemaSerializer

Are you willing to submit a PR?

• I'm willing to submit a PR!