Repository navigation
GH-32949: [Go] REE Array IPC read/write - #14223
Conversation
|
CC @zagto |
|
|
d208d59 to
fe970bd
Compare
3519414 to
07570da
Compare
github-actions
Bot
removed
Component: C++
Component: Documentation
Component: Python
Component: R
labels
3e9f906 to
3be3650
Compare
zeroshade
changed the title
|
|
zeroshade
changed the title
westonpace
left a comment
westonpace
left a comment
There was a problem hiding this comment.
A few questions, mostly probably from my lack of knowledge around the integration tests.
There was a problem hiding this comment.
As i mentioned in my comment about starting on a non-zero value, it's technically meaningless and incorrect to start on a 0 value for run-ends, the min-value should be 1 since run-ends are always 1 past the last index. Run-ends should never start on a 0.
Co-authored-by: Weston Pace <[email protected]>
Co-authored-by: zagto <[email protected]>
Co-authored-by: Antoine Pitrou <[email protected]>
Co-authored-by: Antoine Pitrou <[email protected]>
Co-authored-by: Antoine Pitrou <[email protected]>
14b953f to
b74e667
Compare
|
I'll merge this EOD if no one has any objections. |
| // RunEnds: [ 1, 2, 4, 6, 10, 1000, 1750, 2000 ] | ||
| // Values: [ "a", "b", "c", "d", "e", "f", "g", "h" ] | ||
| // | ||
| // LogicalValuesArray will return the following array: |
There was a problem hiding this comment.
The C++ implementation has an iterator class that allows a zero-copy iteration over the runs that match the logical (offset, length) slice. Do you have a similar thing in Go or is this function used instead?
There was a problem hiding this comment.
This function is used for producing the slice that is used for writing the IPC data. It's a zero-copy slice.
In encoded.go there's MergedRuns which provides a similar function to your iterator class which provides zero-copy iteration of runs using offset/length (and for finding common runs between two REE arrays). The only a time a copy happens is when calling LogicalRunEndsArray when the offset is non-zero because it needs to modify the actual run-end values to be zero-adjusted.
|
Benchmark runs are scheduled for baseline = 9e7b79b and contender = 9b4c972. 9b4c972 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Uh oh!
There was an error while loading. Please reload this page.