-
Notifications
You must be signed in to change notification settings - Fork 3k
Description
After #9008 is merged, it becomes possible for an Iceberg agent to create schemas with timestamp_ns columns. The behavior of Iceberg agents not aware of this new type is undefined.
A simple solution: new types require format-version=3
Originally posted by @nastra in #9008 (comment)
Given that this type is for v3, wouldn't a v2 writer just blindly write this new type if it's being used anywhere and thus break forward compability?
@nastra I don't think I understand your question, would you mind sharing an example?
It seems to me that any writer that doesn't recognize a particular type (in this case
timestamp_nsortimestamptz_ns) should fail when it reads the schema in the table metadata. How does the writer "blindly" write to an unimplemented type?
@jacobmarble currently nothing is preventing this new type to be written with a V2 writer, meaning that one could use this new type in a schema and write it with a V2 writer. Now an older Iceberg client tries to read this using a V2 reader and will fail.