Skip to content

ARROW-17642: [C++] Add ordered aggregation #14352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rtpsw wants to merge 14 commits into from
Closed

ARROW-17642: [C++] Add ordered aggregation #14352

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
8abc366
ARROW-17642: [C++] Add ordered aggregation
rtpsw Oct 8, 2022
9af5d1a
aggregate-node support
rtpsw Oct 10, 2022
fd3b3e8
CI fixes
rtpsw Oct 15, 2022
ec4e1ab
remove unused function
rtpsw Oct 15, 2022
0aaed62
add tests
rtpsw Oct 16, 2022
6901b1a
expose streaming GroupBy
rtpsw Oct 17, 2022
ace4ef9
requested fixes
rtpsw Nov 14, 2022
43949ad
requested fixes
rtpsw Nov 20, 2022
c90d2bf
Merge branch 'master' into ARROW-17642
rtpsw Nov 27, 2022
8f280ee
grouping state clean up
rtpsw Nov 28, 2022
60c70ca
Merge branch 'master' into ARROW-17642
rtpsw Dec 24, 2022
c9abbd6
fix merge
rtpsw Dec 24, 2022
26f991c
requested fixes
rtpsw Dec 24, 2022
9940fa2
segmented group-by using plan
rtpsw Jan 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 16 additions & 5 deletions cpp/src/arrow/compute/exec.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,18 +140,29 @@ ExecBatch ExecBatch::Slice(int64_t offset, int64_t length) const {
return out;
}

Result<ExecBatch> ExecBatch::Make(std::vector<Datum> values) {
if (values.empty()) {
Result<ExecBatch> ExecBatch::SelectValues(const std::vector<int>& ids) const {
std::vector<Datum> selected_values;
selected_values.reserve(ids.size());
for (int id : ids) {
if (id < 0 || static_cast<size_t>(id) >= values.size()) {
return Status::Invalid("ExecBatch invalid value selection: ", id);
}
selected_values.push_back(values[id]);
}
return ExecBatch(std::move(selected_values), length);
}

Result<ExecBatch> ExecBatch::Make(std::vector<Datum> values, int64_t length) {
if (values.empty() && length < 0) {
return Status::Invalid("Cannot infer ExecBatch length without at least one value");
}

int64_t length = -1;
for (const auto& value : values) {
if (value.is_scalar()) {
continue;
}

if (length == -1) {
if (length < 0) {
length = value.length();
continue;
}
Expand All @@ -162,7 +173,7 @@ Result<ExecBatch> ExecBatch::Make(std::vector<Datum> values) {
}
}

if (length == -1) {
if (length < 0) {
length = 1;
}

Expand Down
10 changes: 9 additions & 1 deletion cpp/src/arrow/compute/exec.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,13 @@ struct ARROW_EXPORT ExecBatch {

explicit ExecBatch(const RecordBatch& batch);

static Result<ExecBatch> Make(std::vector<Datum> values);
/// Creates an ExecBatch with length-validation.
///
/// If any value is given, then all values must have a common length. If the given
/// length is negative, then the length of the ExecBatch is set to this common length,
/// or to 1 if no values are given. Otherwise, the given length must equal the common
/// length, if any value is given.
static Result<ExecBatch> Make(std::vector<Datum> values, int64_t length = -1);
Copy link
Member

@westonpace westonpace Dec 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naively one looking at this might be confused why you now need both ExecBatch::Make and ExecBatch::ExecBatch since both take a vector of values and a length.

Looking closer it seems the Make function does the extra work of verifying that the length of the datums match the given length.

Could you add some comments explaining this for future readers?


Result<std::shared_ptr<RecordBatch>> ToRecordBatch(
std::shared_ptr<Schema> schema, MemoryPool* pool = default_memory_pool()) const;
Expand Down Expand Up @@ -227,6 +233,8 @@ struct ARROW_EXPORT ExecBatch {

ExecBatch Slice(int64_t offset, int64_t length) const;

Result<ExecBatch> SelectValues(const std::vector<int>& ids) const;

/// \brief A convenience for returning the types from the batch.
std::vector<TypeHolder> GetTypes() const {
std::vector<TypeHolder> result;
Expand Down
Loading