It seems to me that during table creation bigquery client will ignore Parquet FormatOptions.
The issue is in the following code where ExternalTableDefinition class is being turned into proto message ... as you can see, there is no code that checks for parquet format options (as opposed to AVRO, CSV and BIGTABLE format options).
On the other hand, when the code is parsing a proto message to ExternalTableDefinition, the ParquetOptions are being correctly parsed - see here
My expectation to solve this issue is following:
Something similar to the following should be added to ExternalTableDefinition#toExternalDataConfigurationPb
if (getFormatOptions() != null && FormatOptions.PARQUET.equals(getFormatOptions().getType())) {
externalConfigurationPb.setParquetOptions(((ParquetOptions) getFormatOptions()).toPb());
}
Do you think my assessment is correct and this is indeed a bug? Or have i missed something? Thank you
It seems to me that during table creation bigquery client will ignore Parquet FormatOptions.
The issue is in the following code where
ExternalTableDefinitionclass is being turned into proto message ... as you can see, there is no code that checks for parquet format options (as opposed to AVRO, CSV and BIGTABLE format options).On the other hand, when the code is parsing a proto message to
ExternalTableDefinition, the ParquetOptions are being correctly parsed - see hereMy expectation to solve this issue is following:
Something similar to the following should be added to ExternalTableDefinition#toExternalDataConfigurationPb
Do you think my assessment is correct and this is indeed a bug? Or have i missed something? Thank you