You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[C++][Compute] Support casting a smaller struct to a bigger one if extra fields are nullable #44555
Arrow can already cast to a schema with an extra column if that column is nullable but it can't do the same on structs.
More precisely I want the follow unittests to pass
TEST(Cast, StructToBiggerStruct) {
std::vector<std::string> field_names = {"a", "b"};
std::shared_ptr<Array> a, b;
a = ArrayFromJSON(int8(), "[1, 2]");
b = ArrayFromJSON(int8(), "[3, 4]");
ASSERT_OK_AND_ASSIGN(auto src, StructArray::Make({a, b}, field_names));
const auto dest =
arrow::struct_({std::make_shared<Field>("a", int8()),
std::make_shared<Field>("b", int8()),
std::make_shared<Field>("c", int8(), /*nullable=*/false)});
const auto options = CastOptions::Safe(dest);
EXPECT_RAISES_WITH_MESSAGE_THAT(
TypeError,
::testing::HasSubstr("struct fields don't match or are in the wrong order"),
Cast(src, options));
}
TEST(Cast, StructToBiggerNullableStruct) {
std::vector<std::string> field_names = {"a", "b"};
std::shared_ptr<Array> a, b, c;
a = ArrayFromJSON(int8(), "[1, 2]");
b = ArrayFromJSON(int8(), "[3, 4]");
ASSERT_OK_AND_ASSIGN(auto src, StructArray::Make({a, b}, field_names));
const auto type_dest = arrow::struct_(
{std::make_shared<Field>("a", int8()), std::make_shared<Field>("b", int8()),
std::make_shared<Field>("c", int8(), /*nullable=*/true)});
const auto options = CastOptions::Safe(type_dest);
c = ArrayFromJSON(int8(), "[null, null]");
ASSERT_OK_AND_ASSIGN(auto dest, StructArray::Make({a, b, c}, {"a", "b", "c"}));
CheckCast(src, dest);
}
Does this sound like a reasonable thing to support? The motivation comes from delta-io/delta-rs#1610 but I think the fix would require a change to arrow.
This suddenly became a critical problem for me so I ended up implementing something that works, but I'm not really sure if its a good implementation. If the idea seems reasonable I can open a PR.
Activity
Tom-Newton commentedon Oct 29, 2024
This suddenly became a critical problem for me so I ended up implementing something that works, but I'm not really sure if its a good implementation. If the idea seems reasonable I can open a PR.
kou commentedon Oct 30, 2024
@lidavidm What do you think about this? It seems that this is related to #12724 / #31101.
lidavidm commentedon Oct 30, 2024
Yeah, I think in general if it works on tables but not structs (or vice versa) we should make them both work.
GH-44555: [C++][Compute] Allow casting struct to bigger nullable stru…
lidavidm commentedon Nov 25, 2024
Issue resolved by pull request 44587
#44587
GH-45028: [C++][Compute] Allow cast to reorder struct fields (#45246)