Implement tree explain for ArrowFileSink, fix original URL#15206
Implement tree explain for ArrowFileSink, fix original URL#15206alamb merged 3 commits intoapache:mainfrom
tree explain for ArrowFileSink, fix original URL#15206Conversation
github-actions
bot
added
core
| query TT | ||
| explain COPY (VALUES (1, 'foo', 1, '2023-01-01'), (2, 'bar', 2, '2023-01-02'), (3, 'baz', 3, '2023-01-03')) | ||
| TO 'test_files/scratch/explain_tree/1.json'; | ||
| TO '/tmp/1.json'; |
There was a problem hiding this comment.
It will also print my local directory if I use a relative path.👀
There was a problem hiding this comment.
Yeah, we need to avoid printing the entire path. Let me see if I can find some way
There was a problem hiding this comment.
Maybe we can save the original path for display - perhaps we could save the output_url here as a field on DataSinkConfig 🤔
FileSinkConfig {
...
/// The unresolved URL specified by the user
original_url: String
...
| 09)│ rows: 1 │ | ||
| 10)└───────────────────────────┘ | ||
| 03)│ -------------------- │ | ||
| 04)│ file:///tmp/1.json │ |
There was a problem hiding this comment.
Can we also possibly add format: csv ?
| query TT | ||
| explain COPY (VALUES (1, 'foo', 1, '2023-01-01'), (2, 'bar', 2, '2023-01-02'), (3, 'baz', 3, '2023-01-03')) | ||
| TO 'test_files/scratch/explain_tree/1.json'; | ||
| TO '/tmp/1.json'; |
There was a problem hiding this comment.
Yeah, we need to avoid printing the entire path. Let me see if I can find some way
| query TT | ||
| explain COPY (VALUES (1, 'foo', 1, '2023-01-01'), (2, 'bar', 2, '2023-01-02'), (3, 'baz', 3, '2023-01-03')) | ||
| TO 'test_files/scratch/explain_tree/1.json'; | ||
| TO '/tmp/1.json'; |
There was a problem hiding this comment.
Maybe we can save the original path for display - perhaps we could save the output_url here as a field on DataSinkConfig 🤔
FileSinkConfig {
...
/// The unresolved URL specified by the user
original_url: String
...
|
This is so close thank you @irenjj |
| 08)│ format: memory │ | ||
| 09)│ rows: 1 │ | ||
| 10)└───────────────────────────┘ | ||
| 03)│ -------------------- │ |
alamb
changed the title
tree explain for ArrowFileSinktree explain for ArrowFileSink, fix original URL
|
Love it -- thanks again @irenjj |
2 participants
Which issue does this PR close?
ArrowFileSink#15112Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?