Skip to content

Comments

Allow and prefer non-prefixed extra fields for dataprep hook#27039

Merged
potiuk merged 3 commits intoapache:mainfrom
astronomer:dataprep-no-prefix
Oct 28, 2022
Merged

Allow and prefer non-prefixed extra fields for dataprep hook#27039
potiuk merged 3 commits intoapache:mainfrom
astronomer:dataprep-no-prefix

Conversation

@dstandish
Copy link
Contributor

From 2.3 we no longer need this convention for web UI custom fields. Just cleaning up the codebase to generally not use this pattern.

Copy link
Contributor

@ferruzzi ferruzzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a non-blocking suggestion, otherwise LGTM

Comment on lines 34 to 35
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
backcompat_prefix = "extra__dataprep__"
if field_name.startswith('extra_'):
backcompat_prefix = "extra__dataprep__"
if field_name.startswith(backcompat_prefix):

@dstandish dstandish changed the title No extra prefix required for dataprep hook Allow and prefer non-prefixed extra fields for dataprep hook Oct 22, 2022
From 2.3 we no longer need this convention for web UI custom fields.  Just cleaning up the codebase to generally not use this pattern.
Copy link
Contributor

@ferruzzi ferruzzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a comment. If there is a good reason for the lack of standardization, then carry on. Otherwise I'd prefer (non-blocking) to see consistency when possible.

def _get_field(extras: dict, field_name: str):
"""Get field from extra, first checking short name, then for backcompat we check for prefixed name."""
backcompat_prefix = "extra__dataprep__"
if field_name.startswith("extra_"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In some of these PRs you are using "extra_", some use "extra__" (two underscores), and others are using backcompat_prefix.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No reason just someone suggested adding double underscore and I simply didn't get to all of them... I can do a cleanup pr after all the dust is settled

Copy link
Contributor

@ferruzzi ferruzzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-binding approval

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers kind:documentation provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants