Fixed appending to tables by adding filtering of None rows#356
Merged
Conversation
william-conti
changed the title
Codecov Report
@@ Coverage Diff @@
## main #356 +/- ##
==========================================
+ Coverage 83.47% 83.80% +0.32%
==========================================
Files 30 30
Lines 2269 2272 +3
Branches 395 400 +5
==========================================
+ Hits 1894 1904 +10
+ Misses 290 284 -6
+ Partials 85 84 -1
|
nfx
requested changes
Oct 2, 2023
Collaborator
nfx
left a comment
nfx
left a comment
There was a problem hiding this comment.
looks great! few additions, though
william-conti
force-pushed
the
fix_empty_objects
branch
from
8091d5f to
10dede4
Compare
nfx
changed the title
None rows
nfx
added a commit
that referenced
this pull request
Oct 3, 2023
* Added `inventory_database` name check during installation ([#275](#275)). * Added a column to `$inventory.tables` to specify if a table might have been synchronised to Unity Catalog already or not ([#306](#306)). * Added a migration state to skip already migrated tables ([#325](#325)). * Fixed appending to tables by adding filtering of `None` rows ([#356](#356)). * Fixed handling of missing but linked cluster policies. ([#361](#361)). * Ignore errors for Redash widgets and queries redeployment during installation ([#367](#367)). * Remove exception and added proper logging for groups in the list that… ([#357](#357)). * Skip group migration when no groups are available after preparation step. ([#363](#363)). * Update databricks-sdk requirement from ~=0.9.0 to ~=0.10.0 ([#362](#362)).
Merged
nfx
added a commit
that referenced
this pull request
Oct 3, 2023
* Added `inventory_database` name check during installation ([#275](#275)). * Added a column to `$inventory.tables` to specify if a table might have been synchronised to Unity Catalog already or not ([#306](#306)). * Added a migration state to skip already migrated tables ([#325](#325)). * Fixed appending to tables by adding filtering of `None` rows ([#356](#356)). * Fixed handling of missing but linked cluster policies. ([#361](#361)). * Ignore errors for Redash widgets and queries redeployment during installation ([#367](#367)). * Remove exception and added proper logging for groups in the list that… ([#357](#357)). * Skip group migration when no groups are available after preparation step. ([#363](#363)). * Update databricks-sdk requirement from ~=0.9.0 to ~=0.10.0 ([#362](#362)).
zpappa
pushed a commit
that referenced
this pull request
Oct 4, 2023
* Added `inventory_database` name check during installation ([#275](#275)). * Added a column to `$inventory.tables` to specify if a table might have been synchronised to Unity Catalog already or not ([#306](#306)). * Added a migration state to skip already migrated tables ([#325](#325)). * Fixed appending to tables by adding filtering of `None` rows ([#356](#356)). * Fixed handling of missing but linked cluster policies. ([#361](#361)). * Ignore errors for Redash widgets and queries redeployment during installation ([#367](#367)). * Remove exception and added proper logging for groups in the list that… ([#357](#357)). * Skip group migration when no groups are available after preparation step. ([#363](#363)). * Update databricks-sdk requirement from ~=0.9.0 to ~=0.10.0 ([#362](#362)).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR aims to fix #297 and #346
It adds a utility method to filter rows that have a column containing None, this will help Crawlers to not throw an error when a column is None.
it also checks if the column in the class is Nullable or not, if it's nullable and the value is None, it's ignored