Skip to content
This repository was archived by the owner on Mar 23, 2026. It is now read-only.

fix: external table definition parquet format options#2535

Merged
Neenu1995 merged 6 commits intogoogleapis:mainfrom
jbeluch:main
Mar 7, 2023
Merged

fix: external table definition parquet format options#2535
Neenu1995 merged 6 commits intogoogleapis:mainfrom
jbeluch:main

Conversation

@jbeluch
Copy link
Copy Markdown
Contributor

@jbeluch jbeluch commented Feb 19, 2023

Fixes #2146

@jbeluch jbeluch requested review from a team and loferris February 19, 2023 01:08
@product-auto-label product-auto-label Bot added size: s Pull request size is small. api: bigquery Issues related to the googleapis/java-bigquery API. labels Feb 19, 2023
@shollyman shollyman requested review from Neenu1995 and removed request for loferris February 22, 2023 22:18
@Neenu1995 Neenu1995 added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Feb 23, 2023
@gcf-owl-bot gcf-owl-bot Bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Feb 23, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Feb 23, 2023
@gcf-owl-bot gcf-owl-bot Bot requested a review from a team February 23, 2023 20:53
@jbeluch
Copy link
Copy Markdown
Contributor Author

jbeluch commented Feb 27, 2023

Hi @Neenu1995 , is there an expected timeline for this type of change to be merged? Thanks.

@Neenu1995 Neenu1995 changed the title fix external table definition parquet format options fix: external table definition parquet format options Mar 2, 2023
@Neenu1995 Neenu1995 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Mar 2, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Mar 2, 2023
@Neenu1995
Copy link
Copy Markdown
Contributor

Neenu1995 commented Mar 2, 2023

Tests fail:

java.lang.ClassCastException: com.google.cloud.bigquery.FormatOptions cannot be cast to com.google.cloud.bigquery.ParquetOptions
	at com.google.cloud.bigquery.ExternalTableDefinition.toExternalDataConfigurationPb(ExternalTableDefinition.java:315)
	at com.google.cloud.bigquery.ExternalTableDefinition.toPb(ExternalTableDefinition.java:283)
	at com.google.cloud.bigquery.TableInfo.toPb(TableInfo.java:511)
	at com.google.cloud.bigquery.BigQueryImpl.create(BigQueryImpl.java:290)
	at com.google.cloud.bigquery.it.ITBigQueryTest.testCreateExternalTableWithReferenceFileSchemaParquet(ITBigQueryTest.java:5175)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

@Neenu1995 Neenu1995 added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Mar 3, 2023
@jbeluch
Copy link
Copy Markdown
Contributor Author

jbeluch commented Mar 6, 2023

Tests fail:

java.lang.ClassCastException: com.google.cloud.bigquery.FormatOptions cannot be cast to com.google.cloud.bigquery.ParquetOptions
	at com.google.cloud.bigquery.ExternalTableDefinition.toExternalDataConfigurationPb(ExternalTableDefinition.java:315)
	at com.google.cloud.bigquery.ExternalTableDefinition.toPb(ExternalTableDefinition.java:283)
	at com.google.cloud.bigquery.TableInfo.toPb(TableInfo.java:511)
	at com.google.cloud.bigquery.BigQueryImpl.create(BigQueryImpl.java:290)
	at com.google.cloud.bigquery.it.ITBigQueryTest.testCreateExternalTableWithReferenceFileSchemaParquet(ITBigQueryTest.java:5175)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

thanks @Neenu1995, i missed running them the first time. ive made an additional fix so the IT test passes now. how should i handle squashing my commits? there was a bot commit on the branch so i didnt want to force push.

Neenu1995
Neenu1995 reviewed Mar 6, 2023
View reviewed changes
Comment thread google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/FormatOptions.java
}

/** Default options for PARQUET format. */
public static FormatOptions parquet() {
Copy link
Copy Markdown
Contributor

@Neenu1995 Neenu1995 Mar 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep the return type as is.
Changing it will break existing customers.

Copy link
Copy Markdown
Contributor Author

@jbeluch jbeluch Mar 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed.

should i handle squashing commits myself or is it done at merge time?

Copy link
Copy Markdown
Contributor

@Neenu1995 Neenu1995 Mar 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jbeluch. The commits should be updated now.

@Neenu1995 Neenu1995 added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Mar 6, 2023
@gcf-owl-bot gcf-owl-bot Bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 6, 2023
@yoshi-kokoro yoshi-kokoro removed kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Mar 6, 2023
@Neenu1995 Neenu1995 added kokoro:force-run Add this label to force Kokoro to re-run the tests. automerge Merge the pull request once unit tests and other checks pass. labels Mar 7, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Mar 7, 2023
@Neenu1995 Neenu1995 added the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 7, 2023
@gcf-owl-bot gcf-owl-bot Bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 7, 2023
@Neenu1995 Neenu1995 removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Mar 7, 2023
@Neenu1995 Neenu1995 merged commit eb45973 into googleapis:main Mar 7, 2023
@gcf-merge-on-green gcf-merge-on-green Bot removed the automerge Merge the pull request once unit tests and other checks pass. label Mar 7, 2023
@release-please release-please Bot mentioned this pull request Mar 7, 2023
Merged
gcf-merge-on-green Bot pushed a commit that referenced this pull request Mar 8, 2023
@release-please
chore(main): release 2.23.2 (#2556)
19385e6 
🤖 I have created a release *beep* *boop*
---


## [2.23.2](https://togithub.com/googleapis/java-bigquery/compare/v2.23.1...v2.23.2) (2023-03-07)


### Bug Fixes

* External table definition parquet format options ([#2535](https://togithub.com/googleapis/java-bigquery/issues/2535)) ([eb45973](https://togithub.com/googleapis/java-bigquery/commit/eb4597314fad72bbdb666a832f0f15f732f40817))


### Documentation

* Remove stale snippet comment ([#2555](https://togithub.com/googleapis/java-bigquery/issues/2555)) ([a71b1b2](https://togithub.com/googleapis/java-bigquery/commit/a71b1b2abb568bd1ed088f7bd4b77f93a68ec95e))

---
This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
This was referenced Mar 14, 2023
Closed
Closed
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer

@Neenu1995 Neenu1995 Neenu1995 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

api: bigquery Issues related to the googleapis/java-bigquery API. size: s Pull request size is small.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Parquet format options are ignored during BQ table creation

3 participants

@jbeluch @Neenu1995 @yoshi-kokoro