Skip to content

Functionality to walk a config.Value tree#1081

Merged
pietern merged 3 commits intomainfrom
value-path
Dec 22, 2023
Merged

Functionality to walk a config.Value tree#1081
pietern merged 3 commits intomainfrom
value-path

Conversation

@pietern
Copy link
Copy Markdown
Contributor

@pietern pietern commented Dec 20, 2023

Changes

This change adds:

  • A config.Walk function to walk a configuration tree
  • A config.Path type to represent a value's path inside a tree
  • Functions to create a config.Path from a string, or convert one to a string

Tests

Additional unit tests with full coverage.

@pietern
Functionality to walk a config.Value tree
875454d 
This change adds:
* A `config.Walk` function to walk a configuration tree
* A `config.Path` type to represent a value's path inside a tree
* Functions to create a `config.Path` from a string, or convert one to a string
@pietern
Copy link
Copy Markdown
Contributor Author

pietern commented Dec 20, 2023

This change is in support of another WIP change to start using config.Value for our bundle configuration.

I broke this out into a separate PR to keep the main one reviewable.

andrewnester
andrewnester reviewed Dec 21, 2023
View reviewed changes
libs/config/path_string_test.go
},
{
// But after an index there must be a dot.
input: "foo[1]bar",
Copy link
Copy Markdown
Contributor

@andrewnester andrewnester Dec 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spaces are not allowed in the path, is it correct?

Copy link
Copy Markdown
Contributor Author

@pietern pietern Dec 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not covered in the parsing code. They will become part of the keys where applicable (i.e. in the middle of a key, or after an index).

Do you think it's worth defining rules for these?

Copy link
Copy Markdown
Contributor

@andrewnester andrewnester Dec 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can follow up on this later including addressing dots in the key names (if we have a case for this)

@pietern pietern requested a review from andrewnester December 22, 2023 09:59
andrewnester
andrewnester reviewed Dec 22, 2023
View reviewed changes
libs/config/path_string.go
// - foo.bar[1]
// - foo.bar[1][2]
// - .
func NewPathFromString(input string) (Path, error) {
Copy link
Copy Markdown
Contributor

@andrewnester andrewnester Dec 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realised that since this will be eventually used for working with real config, we might run into issues where the key itself can contain . like for example

 "spark_conf": {
          "spark.databricks.cluster.profile": "singleNode",
          "spark.master": "local[*, 4]"
        },

What if we format the path in the following way instead

"foo"[1]."bar"

Then if the key has spaces or dots the path will be less confusing, like

"foo.foo"[1]."bar baz"

Copy link
Copy Markdown
Contributor Author

@pietern pietern Dec 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great suggestion -- we can do this later if/when we need to.

andrewnester
andrewnester approved these changes Dec 22, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@andrewnester andrewnester left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chatted offline: we can followup on dots and spaces in keys later if needed

@pietern pietern added this pull request to the merge queue Dec 22, 2023
Merged via the queue into main with commit a1297d7 Dec 22, 2023
@pietern pietern deleted the value-path branch December 22, 2023 10:43
pietern added a commit that referenced this pull request Jan 11, 2024
@pietern
Release v0.212.0
6dbcca8 
CLI:
 * Tweak command groups in CLI help ([#1092](#1092)).
 * Fixed storage-credentials list command in text output ([#1094](#1094)).
 * Allow account client auth with environment variables when no `.databrickscfg` file present ([#1097](#1097)).
 * Always log with text format by default ([#1096](#1096)).

Bundles:
 * Added exec.NewCommandExecutor to execute commands with correct interpreter ([#1075](#1075)).
 * Use resource key as name in permissions code ([#1087](#1087)).
 * Add support for reprompts if user input does not match template schema ([#946](#946)).
 * Improve experience for multiple builtin templates ([#1052](#1052)).
 * Improve error when bundle root is not writable ([#1093](#1093)).
 * Change recommended production deployment path from `/Shared` to `/Users` ([#1091](#1091)).
 * Allow referencing bundle resources by name ([#872](#872)).
 * Make libs/exec fallback to `sh` if `bash` cannot be found ([#1114](#1114)).

Internal:
 * Functionality to walk a `config.Value` tree ([#1081](#1081)).
 * Rename libs/config -> libs/dyn ([#1086](#1086)).
 * Added codecov integration on code push ([#1095](#1095)).
 * Consolidate functions to convert `dyn.Value` to native types ([#1100](#1100)).
 * Define constant for the invalid `dyn.Value` ([#1101](#1101)).

API Changes:
 * Changed `databricks serving-endpoints update-config` command with new required argument order.
 * Changed `databricks account ip-access-lists create` command with new required argument order.
 * Changed `databricks account ip-access-lists replace` command with new required argument order.
 * Changed `databricks account ip-access-lists update` command with new required argument order.
 * Changed `databricks ip-access-lists create` command with new required argument order.
 * Changed `databricks ip-access-lists replace` command with new required argument order.
 * Changed `databricks ip-access-lists update` command with new required argument order.

OpenAPI commit a7a9dc025bb80303e676bf3708942c6aa06689f1 (2024-01-04)

Dependency updates:
 * Bump github.com/hashicorp/terraform-exec from 0.19.0 to 0.20.0 ([#1088](#1088)).
 * Bump go.uber.org/mock from 0.3.0 to 0.4.0 ([#1089](#1089)).
 * Bump github.com/hashicorp/terraform-json from 0.18.0 to 0.20.0 ([#1090](#1090)).
 * Bump golang.org/x/term from 0.15.0 to 0.16.0 ([#1103](#1103)).
 * Bump github.com/cloudflare/circl from 1.3.3 to 1.3.7 ([#1105](#1105)).
 * Bump golang.org/x/sync from 0.5.0 to 0.6.0 ([#1104](#1104)).
 * Upgrade golang.org/x/crypto to v0.17.0 in internal module ([#1110](#1110)).
 * Upgrade Go SDK to v0.29.0 ([#1111](#1111)).
@pietern pietern mentioned this pull request Jan 11, 2024
Merged
github-merge-queue bot pushed a commit that referenced this pull request Jan 11, 2024
@pietern
Release v0.212.0 (#1116)
51c64ef 
CLI:
* Tweak command groups in CLI help
([#1092](#1092)).
* Fixed storage-credentials list command in text output
([#1094](#1094)).
* Allow account client auth with environment variables when no
`.databrickscfg` file present
([#1097](#1097)).
* Always log with text format by default
([#1096](#1096)).

Bundles:
* Added exec.NewCommandExecutor to execute commands with correct
interpreter ([#1075](#1075)).
* Use resource key as name in permissions code
([#1087](#1087)).
* Add support for reprompts if user input does not match template schema
([#946](#946)).
* Improve experience for multiple builtin templates
([#1052](#1052)).
* Improve error when bundle root is not writable
([#1093](#1093)).
* Change recommended production deployment path from `/Shared` to
`/Users` ([#1091](#1091)).
* Allow referencing bundle resources by name
([#872](#872)).
* Make libs/exec fallback to `sh` if `bash` cannot be found
([#1114](#1114)).

Internal:
* Functionality to walk a `config.Value` tree
([#1081](#1081)).
* Rename libs/config -> libs/dyn
([#1086](#1086)).
* Added codecov integration on code push
([#1095](#1095)).
* Consolidate functions to convert `dyn.Value` to native types
([#1100](#1100)).
* Define constant for the invalid `dyn.Value`
([#1101](#1101)).

API Changes:
* Changed `databricks serving-endpoints update-config` command with new
required argument order.
* Changed `databricks account ip-access-lists create` command with new
required argument order.
* Changed `databricks account ip-access-lists replace` command with new
required argument order.
* Changed `databricks account ip-access-lists update` command with new
required argument order.
* Changed `databricks ip-access-lists create` command with new required
argument order.
* Changed `databricks ip-access-lists replace` command with new required
argument order.
* Changed `databricks ip-access-lists update` command with new required
argument order.

OpenAPI commit a7a9dc025bb80303e676bf3708942c6aa06689f1 (2024-01-04)

Dependency updates:
* Bump github.com/hashicorp/terraform-exec from 0.19.0 to 0.20.0
([#1088](#1088)).
* Bump go.uber.org/mock from 0.3.0 to 0.4.0
([#1089](#1089)).
* Bump github.com/hashicorp/terraform-json from 0.18.0 to 0.20.0
([#1090](#1090)).
* Bump golang.org/x/term from 0.15.0 to 0.16.0
([#1103](#1103)).
* Bump github.com/cloudflare/circl from 1.3.3 to 1.3.7
([#1105](#1105)).
* Bump golang.org/x/sync from 0.5.0 to 0.6.0
([#1104](#1104)).
* Upgrade golang.org/x/crypto to v0.17.0 in internal module
([#1110](#1110)).
* Upgrade Go SDK to v0.29.0
([#1111](#1111)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewnester andrewnester andrewnester approved these changes

@shreyas-goenka shreyas-goenka Awaiting requested review from shreyas-goenka shreyas-goenka is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@pietern @andrewnester