Changed service template to correctly handle required positional arguments by andrewnester · Pull Request #405 · databricks/cli

andrewnester · 2023-05-24T15:48:03Z

Changes

Change 1

Previously codegen considered that command has positional arguments only if all arguments are primitives.
With this change, we consider command to have positional arguments if all required arguments are primitives.

It helps to solve thew issue when running command like cli secrets create-scope, cli users create with no passed arguments

Change 2

It also makes args optional if there is a prompt for it or JSON can be accepted instead.
All 3 command below will yield the same valid results

cli clusters events --json '{"cluster_id": "1029-xxxx"}'
...
cli clusters events 1029-xxxx
...
cli clusters events
Search: █
? The ID of the cluster to retrieve events about: 
  10.1 
  10.5 
  10xlibs
  12.2 LTS
↓ 13.x

Change 3

Also now it always allow --json flag and ignore all positional arguments in this case

cli account private-access create --json '{}'
Error: accepts 2 arg(s), received 0

cli account private-access create --json '{}'
Use the arrow keys to navigate: ↓ ↑ → ← 
? ~/.databrickscfg profile:

Note: these code is generated with code generator containing this fix
databricks/databricks-sdk-go#403

Tests

[DEBUG] Test execution directory:  /Users/andrew.nester/cli
2023/05/26 14:02:25 [INFO] ✅ TestAccApiGet (0.630s)
2023/05/26 14:02:26 [INFO] ✅ TestAccClustersList (0.820s)
2023/05/26 14:02:26 [INFO] ✅ TestAccClustersGet (0.810s)
...

Running cli secrets create-scope, cli users create and etc with no arguments.

No panic, instead CLI correctly prompts potential input or shows correct error message.

Before

andrew.nester@HFW9Y94129 multiples-tasks % cli users create        
panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
github.com/databricks/cli/cmd/workspace/users.glob..func1(0x101552a60?, {0x10159c6e0, 0x0, 0x0?})
        /Users/andrew.nester/cli/cmd/workspace/users/users.go:71 +0x1c0
github.com/spf13/cobra.(*Command).execute(0x101552a60, {0x10159c6e0, 0x0, 0x0})
        /Users/andrew.nester/cli/vendor/github.com/spf13/cobra/command.go:940 +0x5c8
github.com/spf13/cobra.(*Command).ExecuteC(0x10151a680)
        /Users/andrew.nester/cli/vendor/github.com/spf13/cobra/command.go:1068 +0x35c
github.com/spf13/cobra.(*Command).ExecuteContextC(...)
        /Users/andrew.nester/cli/vendor/github.com/spf13/cobra/command.go:1001
github.com/databricks/cli/cmd/root.Execute()
        /Users/andrew.nester/cli/cmd/root/root.go:81 +0x6c
main.main()
        /Users/andrew.nester/cli/main.go:18 +0x1c

After

andrew.nester@HFW9Y94129 multiples-tasks % cli users create       
Error: /2.0/preview/scim/v2/Users is only accessible by admins

…ments

pietern

Thanks -- this LGTM except for the create case. There the prompt is unexpected.

Can you investigate why this happens?

andrewnester · 2023-05-25T10:37:51Z

@pietern see the fix and explanation why code generator worked that way here: databricks/databricks-sdk-go#403

cmd/account/groups/groups.go

cmd/account/workspaces/workspaces.go

cmd/workspace/clusters/clusters.go

pietern

I spot checked most generated commands and looks good!

Could you see if you can keep Args: cobra.MaximumNArgs(), for commands that have an optional argument? It would we 1 in cases where we can prompt.

pietern · 2023-05-25T14:38:00Z

.codegen/service.go.tmpl

 {{- $serviceHasNamedIdMap := and .Service.List .Service.List.NamedIdMap -}}
 {{- $hasIdPrompt := and $hasSinglePosArg $serviceHasNamedIdMap -}}
 {{- $wait := and .Wait (and (not .IsCrudRead) (not (eq .SnakeName "get_run"))) -}}
+{{- $canAcceptJson := and .Request (not .Request.IsOnlyPrimitiveFields) -}}


Won't these still be able to accept json if the request is only primitive fields?

Technically we can allow JSON to be accepted in all cases. Right now if all fields are primitive we don't add --json flag and instead add a specific flag for each of the fields.

We can change it to always accept JSON though

pietern · 2023-05-26T12:27:37Z

.codegen/service.go.tmpl

    {{end}}
-	Annotations: map[string]string{},{{if and (not $hasIdPrompt) $hasPosArgs }}
-	Args: cobra.ExactArgs({{len .Request.RequiredFields}}),{{end}}
+	Annotations: map[string]string{},{{if $hasRequiredArgs }}


FWIW, you can use {{- to strip preceding whitespace (including newlines).

See https://pkg.go.dev/text/template#hdr-Text_and_spaces

This means you could put the if on the next line for better readability.

pietern · 2023-05-26T12:29:27Z

cmd/workspace/query-history/query-history.go

+		}
+		return check(cmd, args)
+	},
+	PreRunE: root.MustWorkspaceClient,


This doesn't look fmt'ed.

## Changes CLI: * Add directory tracking to sync ([#425](#425)). * Add fs cat command for dbfs files ([#430](#430)). * Add fs ls command for dbfs ([#429](#429)). * Add fs mkdirs command for dbfs ([#432](#432)). * Add fs rm command for dbfs ([#433](#433)). * Add installation instructions ([#458](#458)). * Add new line to cmdio JSON rendering ([#443](#443)). * Add profile on `databricks auth login` ([#423](#423)). * Add readable console logger ([#370](#370)). * Add workspace export-dir command ([#449](#449)). * Added secrets input prompt for secrets put-secret command ([#413](#413)). * Added spinner when loading command prompts ([#420](#420)). * Better error message if can not load prompts ([#437](#437)). * Changed service template to correctly handle required positional arguments ([#405](#405)). * Do not generate prompts for certain commands ([#438](#438)). * Do not prompt for List methods ([#411](#411)). * Do not use FgWhite and FgBlack for terminal output ([#435](#435)). * Skip path translation of job task for jobs with a Git source ([#404](#404)). * Tweak profile prompt ([#454](#454)). * Update with the latest Go SDK ([#457](#457)). * Use cmdio in version command for `--output` flag ([#419](#419)). Bundles: * Check for nil environment before accessing it ([#453](#453)). Dependencies: * Bump github.com/hashicorp/terraform-json from 0.16.0 to 0.17.0 ([#459](#459)). * Bump github.com/mattn/go-isatty from 0.0.18 to 0.0.19 ([#412](#412)). Internal: * Add Mkdir and ReadDir functions to filer.Filer interface ([#414](#414)). * Add Stat function to filer.Filer interface ([#421](#421)). * Add check for path is a directory in filer.ReadDir ([#426](#426)). * Add fs.FS adapter for the filer interface ([#422](#422)). * Add implementation of filer.Filer for local filesystem ([#460](#460)). * Allow equivalence checking of filer errors to fs errors ([#416](#416)). * Fix locker integration test ([#417](#417)). * Implement DBFS filer ([#139](#139)). * Include recursive deletion in filer interface ([#442](#442)). * Make filer.Filer return fs.DirEntry from ReadDir ([#415](#415)). * Speed up sync integration tests ([#428](#428)).

…vided (#535) ## Changes When there are positional required parameters in the command which can't be unmarshalled from JSON, we should require them despite the fact `--json` flag is provided. The reason is that for some of the command, for example, `databricks groups patch ID` these arguments are actually path arguments in API and can't be set as part of `--json` body provided. Original change which introduced this ignore logic is here: #405 Fixes #533, #537 Note: Code generation is based on the change in this PR: databricks/databricks-sdk-go#536 ## Tests 1. Running `cli groups patch 123 --json {...}` works correctly Backward compatibility tests with previous changes from #405 1. `cli clusters events --json '{"cluster_id": "1029-xxxx"}'` - works, returns list of events 2. `cli clusters events 1029-xxxx` - works, returns list of events 3. `cli clusters events` - works, first prompts for Cluster ID and then returns the list of events

Changed service template to correctly handle required positional argu…

4349103

…ments

andrewnester requested review from pietern and shreyas-goenka May 24, 2023 15:48

pietern reviewed May 25, 2023

View reviewed changes

Regenerated with fixed generator

ce86dcc

andrewnester requested a review from pietern May 25, 2023 10:38

pietern reviewed May 25, 2023

View reviewed changes

cmd/account/groups/groups.go Show resolved Hide resolved

cmd/account/workspaces/workspaces.go Outdated Show resolved Hide resolved

cmd/workspace/clusters/clusters.go Outdated Show resolved Hide resolved

Make args optional if there is a prompt or accepts JSON

4f69d6c

andrewnester requested a review from pietern May 25, 2023 13:51

pietern reviewed May 25, 2023

View reviewed changes

This comment was marked as duplicate.

Sign in to view

pietern approved these changes May 25, 2023

View reviewed changes

Always allow json flag

3853759

andrewnester requested a review from pietern May 25, 2023 15:20

Added integration tests

1c91e73

pietern approved these changes May 26, 2023

View reviewed changes

andrewnester merged commit 3c4d6f6 into main May 26, 2023

andrewnester deleted the secrets-panic branch May 26, 2023 12:46

pietern mentioned this pull request Jun 12, 2023

Release v0.100.3 #461

Merged

andrewnester mentioned this pull request Jun 29, 2023

Fixed ignoring required positional parameters when --json flag is provided #535

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changed service template to correctly handle required positional arguments#405

Changed service template to correctly handle required positional arguments#405
andrewnester merged 5 commits intomainfrom
secrets-panic

andrewnester commented May 24, 2023 •

edited

Loading

Uh oh!

pietern left a comment

Uh oh!

andrewnester commented May 25, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pietern left a comment

Uh oh!

pietern May 25, 2023

Uh oh!

andrewnester May 25, 2023

Uh oh!

This comment was marked as duplicate.

Uh oh!

pietern May 26, 2023

Uh oh!

pietern May 26, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

andrewnester commented May 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Change 1

Change 2

Change 3

Tests

Uh oh!

pietern left a comment

Choose a reason for hiding this comment

Uh oh!

andrewnester commented May 25, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pietern left a comment

Choose a reason for hiding this comment

Uh oh!

pietern May 25, 2023

Choose a reason for hiding this comment

Uh oh!

andrewnester May 25, 2023

Choose a reason for hiding this comment

Uh oh!

This comment was marked as duplicate.

Uh oh!

pietern May 26, 2023

Choose a reason for hiding this comment

Uh oh!

pietern May 26, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

andrewnester commented May 24, 2023 •

edited

Loading