Add an experimental dbt-sql template by lennartkats-db · Pull Request #1059 · databricks/cli

lennartkats-db · 2023-12-12T15:01:58Z

Changes

This adds a new dbt-sql template. This work requires the new WorkspaceFS support for dbt tasks.

In this latest revision, I've hidden the new template from the list so we can merge it, iterate over it, and propertly release the template at the right time.

Blockers:

Tests

libs/template/templates/dbt-sql/databricks_template_schema.json

libs/template/templates/dbt-sql/template/{{.project_name}}/profile_template.yml.tmpl

## Changes This enhances the template selection experience a bit as we add more and more built-in templates (like #1051 and #1059): ### New experience: <img width="661" alt="image" src="https://github.com/databricks/cli/assets/58432911/afe3b84d-8a77-47f3-b9c2-f827f7893cd7"> ### Current experience: <img width="265" alt="image" src="https://github.com/databricks/cli/assets/58432911/36f8d568-819f-4920-83b1-fb76109ea3d1"> --------- Co-authored-by: shreyas-goenka <[email protected]>

codecov-commenter · 2024-01-27T17:25:24Z

Codecov Report

Attention: 12 lines in your changes are missing coverage. Please review.

Comparison is base (f70ec35) 52.53% compared to head (e5fb708) 52.51%.

Files	Patch %	Lines
libs/template/helpers.go	14.28%	11 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1059      +/-   ##
==========================================
- Coverage   52.53%   52.51%   -0.02%     
==========================================
  Files         308      308              
  Lines       17160    17175      +15     
==========================================
+ Hits         9015     9020       +5     
- Misses       7481     7489       +8     
- Partials      664      666       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

libs/template/helpers.go

pietern · 2024-02-05T08:03:40Z

libs/template/templates/dbt-sql/databricks_template_schema.json

+        },
+        "http_path": {
+            "type": "string",
+            "pattern": "^/sql/.\\../warehouses/[a-z0-9]+$",


How does this pattern work without + or *?

I'm not sure I understand this question. There is a + in there?

Or maybe you're referring to the \.\\.. part? That matches a version, like 1.0.

libs/template/templates/dbt-sql/databricks_template_schema.json

libs/template/templates/dbt-sql/template/{{.project_name}}/README.md.tmpl

CLI: * Add support for UC Volumes to the `databricks fs` commands ([#1209](#1209)). Bundles: * Use dynamic configuration model in bundles ([#1098](#1098)). * Allow use of variables references in primitive non-string fields ([#1219](#1219)). * Add an experimental default-sql template ([#1051](#1051)). * Add an experimental dbt-sql template ([#1059](#1059)). Internal: * Add fork-user to winget release workflow ([#1214](#1214)). * Use `any` as type for data sources and resources in `tf/schema` ([#1216](#1216)). * Avoid infinite recursion when normalizing a recursive type ([#1213](#1213)). * Fix issue where interpolating a new ref would rewrite unrelated fields ([#1217](#1217)). * Use `dyn.Value` as input to generating Terraform JSON ([#1218](#1218)). API Changes: * Changed `databricks lakehouse-monitors update` command with new required argument order. * Added `databricks online-tables` command group. OpenAPI commit cdd76a98a4fca7008572b3a94427566dd286c63b (2024-02-19) Dependency updates: * Bump Terraform provider to v1.36.2 ([#1215](#1215)). * Bump github.com/databricks/databricks-sdk-go from 0.32.0 to 0.33.0 ([#1222](#1222)).

Add a dbt template

4fee665

lennartkats-db changed the title ~~[WIP] Add a dbt template~~ [WIP] Add a dbt-sql template Dec 12, 2023

lennartkats-db mentioned this pull request Dec 12, 2023

Improve experience for multiple builtin templates #1052

Merged

Use a template for VS Code settings

b3a5ef8

pietern reviewed Dec 15, 2023

View reviewed changes

libs/template/templates/dbt-sql/databricks_template_schema.json Show resolved Hide resolved

libs/template/templates/dbt-sql/template/{{.project_name}}/profile_template.yml.tmpl Outdated Show resolved Hide resolved

lennartkats-db added 2 commits December 19, 2023 10:27

Tweak message

c81d139

Update

c900cff

lennartkats-db changed the title ~~[WIP] Add a dbt-sql template~~ [WIP] [DONTMERGE] Add a dbt-sql template Dec 23, 2023

Add tests

16f26a3

lennartkats-db added 2 commits December 28, 2023 13:32

Merge remote-tracking branch 'databricks/main' into dbt-template

cd52c83

Fix test

d85d4c4

lennartkats-db marked this pull request as ready for review December 28, 2023 12:34

lennartkats-db changed the title ~~[WIP] [DONTMERGE] Add a dbt-sql template~~ [DONTMERGE] Add a dbt-sql template Dec 28, 2023

lennartkats-db requested a review from pietern December 28, 2023 12:35

lennartkats-db added 7 commits January 8, 2024 21:16

Merge remote-tracking branch 'databricks/main' into dbt-template

419bd27

Add template

9030f56

Improve catalog handling

45ea8db

Minor tweaks

0268c88

Update template to use materialized views & streaming tables

94ebd9a

Add conditional

14bc1fa

Improve template

1501298

lennartkats-db force-pushed the dbt-template branch from 62a0b69 to 1501298 Compare January 20, 2024 21:59

lennartkats-db added 4 commits January 22, 2024 09:23

Offer an option to use personal schemas

6fc5ed4

Merge remote-tracking branch 'databricks/main' into dbt-template

220a1ea

Fix ANSI mode

99f920e

Merge remote-tracking branch 'databricks/main' into dbt-template

af0dd6d

pietern added the do-not-merge label Jan 25, 2024

lennartkats-db added 2 commits January 25, 2024 21:17

Don't ask for a "production" schema, just assume "default"

1099eed

Explain mode: development

33c5e91

lennartkats-db changed the title ~~[DONTMERGE] Add a dbt-sql template~~ [DONTMERGE] [WIP] Add a dbt-sql template Jan 27, 2024

lennartkats-db added 4 commits January 27, 2024 23:45

Improve DX with default_catalog helper

de7bd78

Remove from list of templates for now

8e7c6a1

Update README.md

18c6b70

Merge remote-tracking branch 'databricks/main' into dbt-template

a660efa

lennartkats-db changed the title ~~[DONTMERGE] [WIP] Add a dbt-sql template~~ Add a dbt-sql template Jan 28, 2024

lennartkats-db changed the title ~~Add a dbt-sql template~~ Add an experimental dbt-sql template Jan 29, 2024

Mark as experimental

2f52ff1

lennartkats-db removed the do-not-merge label Feb 4, 2024

pietern approved these changes Feb 5, 2024

View reviewed changes

pietern mentioned this pull request Feb 5, 2024

Add an experimental default-sql template #1051

Merged

9 tasks

juliacrawf-db reviewed Feb 13, 2024

View reviewed changes

libs/template/templates/dbt-sql/databricks_template_schema.json Show resolved Hide resolved

juliacrawf-db reviewed Feb 19, 2024

View reviewed changes

libs/template/templates/dbt-sql/template/{{.project_name}}/README.md.tmpl Outdated Show resolved Hide resolved

juliacrawf-db reviewed Feb 19, 2024

View reviewed changes

libs/template/templates/dbt-sql/template/{{.project_name}}/README.md.tmpl Outdated Show resolved Hide resolved

juliacrawf-db reviewed Feb 19, 2024

View reviewed changes

libs/template/templates/dbt-sql/template/{{.project_name}}/README.md.tmpl Outdated Show resolved Hide resolved

lennartkats-db added 3 commits February 19, 2024 09:56

Restore sql-dbt template in hidden form

e041148

Merge remote-tracking branch 'databricks/main' into dbt-template

00bf2fe

Copy-editing

e5fb708

lennartkats-db force-pushed the dbt-template branch from aaec97d to e5fb708 Compare February 19, 2024 09:07

lennartkats-db enabled auto-merge February 19, 2024 09:10

lennartkats-db removed the request for review from PaulCornellDB February 19, 2024 09:12

lennartkats-db added this pull request to the merge queue Feb 19, 2024

Merged via the queue into databricks:main with commit 1c68012 Feb 19, 2024

lennartkats-db deleted the dbt-template branch February 19, 2024 09:20

lennartkats-db mentioned this pull request Feb 20, 2024

Add examples based on default-sql & dbt-sql templates databricks/bundle-examples#23

Merged

andrewnester mentioned this pull request Feb 20, 2024

Release v0.214.0 #1226

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add an experimental dbt-sql template#1059

Add an experimental dbt-sql template#1059
lennartkats-db merged 29 commits intodatabricks:mainfrom
lennartkats-db:dbt-template

lennartkats-db commented Dec 12, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Jan 27, 2024 •

edited

Loading

Uh oh!

Uh oh!

pietern Feb 5, 2024

Uh oh!

lennartkats-db Feb 19, 2024

Uh oh!

lennartkats-db Feb 19, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

lennartkats-db commented Dec 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Tests

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Jan 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

pietern Feb 5, 2024

Choose a reason for hiding this comment

Uh oh!

lennartkats-db Feb 19, 2024

Choose a reason for hiding this comment

Uh oh!

lennartkats-db Feb 19, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

lennartkats-db commented Dec 12, 2023 •

edited

Loading

codecov-commenter commented Jan 27, 2024 •

edited

Loading