Merged
Conversation
Contributor
Author
|
Output: case 1: command is run, directory foo does not exist case 2: foo exists, with a couple of files in it. case 3: foo exists, with a couple of files in it. We use the overwrite flag case 4: output with the json mode enabled |
Contributor
Author
|
github is experiencing an outage, so my latest commits are not in, but they are lint fixes, todo cleanups and making the test prefixes |
pietern
reviewed
Jun 8, 2023
| // 2. Allows use to error out if the path is a directory. This is needed | ||
| // because the Dbfs.Open method on the SDK does not error when the path is | ||
| // a directory | ||
| stat, err := w.Stat(ctx, name) |
Contributor
There was a problem hiding this comment.
Fixed in databricks/databricks-sdk-go#415
I would prefer not adding a stat call for every read if we can avoid it.
Contributor
Author
There was a problem hiding this comment.
I have added a todo with a follow-up issue filed. I would rather not block on a go SDK release or remove test coverage
| defer f.Close() | ||
|
|
||
| // Write content to the local file | ||
| r, err := workspaceFiler.Read(ctx, relPath) |
Contributor
There was a problem hiding this comment.
This means doing 2 stat calls for every object as is.
Contributor
Author
There was a problem hiding this comment.
Well, I think this is OK if a few assumptions hold true:
StatAPI calls do not overload webapp- Latency added is reasonable
The reason I am fine with it, is this should be solved when we have the Files API since it is supposed to be a "DUMB" filesystem (no fancy extension stripping, service throws an error when we try to read a directory)
Right now it's us handing backend problems on the client side causing excessive API calls
Contributor
|
This looks great -- comments are minor. |
pietern
reviewed
Jun 8, 2023
pietern
approved these changes
Jun 8, 2023
Merged
pietern
added a commit
that referenced
this pull request
Jun 12, 2023
## Changes CLI: * Add directory tracking to sync ([#425](#425)). * Add fs cat command for dbfs files ([#430](#430)). * Add fs ls command for dbfs ([#429](#429)). * Add fs mkdirs command for dbfs ([#432](#432)). * Add fs rm command for dbfs ([#433](#433)). * Add installation instructions ([#458](#458)). * Add new line to cmdio JSON rendering ([#443](#443)). * Add profile on `databricks auth login` ([#423](#423)). * Add readable console logger ([#370](#370)). * Add workspace export-dir command ([#449](#449)). * Added secrets input prompt for secrets put-secret command ([#413](#413)). * Added spinner when loading command prompts ([#420](#420)). * Better error message if can not load prompts ([#437](#437)). * Changed service template to correctly handle required positional arguments ([#405](#405)). * Do not generate prompts for certain commands ([#438](#438)). * Do not prompt for List methods ([#411](#411)). * Do not use FgWhite and FgBlack for terminal output ([#435](#435)). * Skip path translation of job task for jobs with a Git source ([#404](#404)). * Tweak profile prompt ([#454](#454)). * Update with the latest Go SDK ([#457](#457)). * Use cmdio in version command for `--output` flag ([#419](#419)). Bundles: * Check for nil environment before accessing it ([#453](#453)). Dependencies: * Bump github.com/hashicorp/terraform-json from 0.16.0 to 0.17.0 ([#459](#459)). * Bump github.com/mattn/go-isatty from 0.0.18 to 0.0.19 ([#412](#412)). Internal: * Add Mkdir and ReadDir functions to filer.Filer interface ([#414](#414)). * Add Stat function to filer.Filer interface ([#421](#421)). * Add check for path is a directory in filer.ReadDir ([#426](#426)). * Add fs.FS adapter for the filer interface ([#422](#422)). * Add implementation of filer.Filer for local filesystem ([#460](#460)). * Allow equivalence checking of filer errors to fs errors ([#416](#416)). * Fix locker integration test ([#417](#417)). * Implement DBFS filer ([#139](#139)). * Include recursive deletion in filer interface ([#442](#442)). * Make filer.Filer return fs.DirEntry from ReadDir ([#415](#415)). * Speed up sync integration tests ([#428](#428)).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
This PR:
Tests
Integration tests and manually