Skip to content

GH-32946: [Go] Implement REE Array and Compare - #14111

Merged
zeroshade merged 4 commits into
apache:masterfrom
zeroshade:rle-initial
Jan 19, 2023
Merged

zeroshade merged 4 commits into
apache:masterfrom
zeroshade:rle-initial

Conversation

@zeroshade

@zeroshade zeroshade commented Sep 13, 2022 •

Copy link
Copy Markdown
Member

@zeroshade

Copy link
Copy Markdown
Member Author

CC @zagto

@github-actions

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

Comment thread go/arrow/array/builder.go Resolved
@zeroshade zeroshade changed the title ARROW-17708: [Go] Implement RLE Array and Compare ARROW-17708: [Go] Implement REE Array and Compare Jan 17, 2023
@zeroshade
zeroshade marked this pull request as ready for review January 17, 2023 19:38
@zeroshade
zeroshade requested a review from lidavidm January 17, 2023 19:44

@lidavidm lidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM, though I need to look at that merged runs helper more closely

Comment thread go/arrow/array/builder.go Resolved
Comment thread go/arrow/compute/internal/exec/span.go Resolved
Comment thread go/arrow/encoded/ree_utils.go Resolved

@lidavidm lidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: PR title needs updating to the GitHub issue

case arrow.INT16:
runEnds := arrow.Int16Traits.CastFromBytes(data.Buffers()[1].Bytes())
runEnds = runEnds[data.Offset() : data.Offset()+data.Len()]
return sort.Search(len(runEnds), func(i int) bool { return runEnds[i] > int16(logicalOffset) })

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. Should we explicitly check for overflow? I suppose that's better left to a validation function (to ensure the array length isn't inconsistent).

I suppose more interestingly, a REE array with 16-bit run ends can only store 32767 elements.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, I probably wouldn't check for overflow here. Currently the Array types don't have an equivalent Validation method like the C++ version does, it's on my list of stuff to do but it's fairly low in priority. Right now it's kinda left to the consumer to ensure they use the correct type to not worry about overflows.

@zeroshade zeroshade changed the title ARROW-17708: [Go] Implement REE Array and Compare GH-32946: [Go] Implement REE Array and Compare Jan 18, 2023
@github-actions

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #32946 has been automatically assigned in GitHub to PR creator.

@zeroshade

Copy link
Copy Markdown
Member Author

If there's no further comments before EOD today i'll merge this.

@zeroshade
zeroshade merged commit 9a13734 into apache:master Jan 19, 2023
@ursabot

ursabot commented Jan 20, 2023

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = 7579dbd and contender = 9a13734. 9a13734 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed] test-mac-arm
[Failed] ursa-i9-9960x
[Failed] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 9a137345 ec2-t3-xlarge-us-east-2
[Failed] 9a137345 test-mac-arm
[Failed] 9a137345 ursa-i9-9960x
[Failed] 9a137345 ursa-thinkcentre-m75q
[Finished] 7579dbdd ec2-t3-xlarge-us-east-2
[Failed] 7579dbdd test-mac-arm
[Failed] 7579dbdd ursa-i9-9960x
[Failed] 7579dbdd ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Go] Implement RLE Array and Compare

4 participants