Skip to content

Bundle watcher#892

Merged
kartikgupta-db merged 25 commits intobundle-integfrom
bundle-watcher
Oct 27, 2023
Merged

Bundle watcher#892
kartikgupta-db merged 25 commits intobundle-integfrom
bundle-watcher

Conversation

@kartikgupta-db
Copy link
Copy Markdown
Contributor

@kartikgupta-db kartikgupta-db commented Oct 6, 2023 •
edited
Loading

Changes

  • Autogenerate BundleSchema.d.ts file (from databricks bundle schema command) to provide typing for the parsed bundle object.
  • Add a watcher for all DABs configuration files
  • Add BundleFileSet object, to handle operations on the full bundle file tree.
  • Used these to add autocompletion for all files included in the root bundle.yaml.

NOTE: This PR currently parses yamls in code. We want to move that to databricks CLI. The implementation can is modular and can be swapped with the CLI calls once we have those features in the CLI.

Tests

  • Unit tests
  • Manual

@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 6, 2023 09:52 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 6, 2023 09:53 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 6, 2023 09:53 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 10:30 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 10:30 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 10:40 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 10:40 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 10:48 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 10:48 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 10:55 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 10:55 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 12:32 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 12:37 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 12:37 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 13:06 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 13:06 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 13:06 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 13, 2023 13:06 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db mentioned this pull request Oct 15, 2023
Closed
@kartikgupta-db kartikgupta-db marked this pull request as ready for review October 16, 2023 10:11
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 16, 2023 10:14 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr October 16, 2023 10:14 — with GitHub Actions Inactive
pietern
pietern reviewed Oct 30, 2023
View reviewed changes
packages/databricks-vscode/src/bundle/bundleAutocompleteProvider.ts
return rootConfigSchemaUri;
}
return undefined;
},
Copy link
Copy Markdown
Contributor

@pietern pietern Oct 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kartikgupta-db Is this registration updated as new bundle configuration files are added, or does this only trigger on extension reload? Not clear to me when this contributor callback is called.

Copy link
Copy Markdown
Contributor Author

@kartikgupta-db kartikgupta-db Oct 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is called everytime a new yaml file is opened/created .The file list update happens only when root file is changed. This is wrong. The fix is here #922

packages/databricks-vscode/src/bundle/bundleAutocompleteProvider.ts
}
if (
bundleFileList.find(
(i) => i.fsPath === Uri.parse(resource).fsPath
Copy link
Copy Markdown
Contributor

@pietern pietern Oct 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can call Uri.parse outside of the lambda to save some cycles.

Copy link
Copy Markdown
Contributor Author

@kartikgupta-db kartikgupta-db Oct 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated here as well #922

packages/databricks-vscode/src/bundle/BundleFileSet.ts
)
).map((i) => Uri.file(i));
}
}
Copy link
Copy Markdown
Contributor

@pietern pietern Oct 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking: this will soon use the bundle validate equivalent output to determine the same, correct?

Copy link
Copy Markdown
Contributor Author

@kartikgupta-db kartikgupta-db Oct 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. I have tried to design it to be easily swappable.

@kartikgupta-db kartikgupta-db mentioned this pull request Oct 30, 2023
Merged
This was referenced Mar 5, 2024
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pietern pietern pietern left review comments

@fjakobs fjakobs Awaiting requested review from fjakobs

@andrewnester andrewnester Awaiting requested review from andrewnester

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kartikgupta-db @pietern