Skip to content

[python/r] Implement resize and tiledbsoma_upgrade_shape#2950

Merged
johnkerl merged 5 commits intomainfrom
kerl/py-r-test-2
Sep 12, 2024
Merged

[python/r] Implement resize and tiledbsoma_upgrade_shape#2950
johnkerl merged 5 commits intomainfrom
kerl/py-r-test-2

Conversation

@johnkerl
Copy link
Copy Markdown
Contributor

@johnkerl johnkerl commented Sep 3, 2024

Issue and/or context: As tracked on issue #2407 / [sc-51048]. Here we implement resize and upgrade_shape for ND arrays. The dataframe mutators are deferred to a separate PR.

Note that the intended Python and R API changes are all agreed on and finalized as described in #2407.

Changes:

Implements and tests the mutators.

Notes for Reviewer:

For a separate PR: needs unit-test cases, including some saved-off arrays (in a tar file, say) lacking the new-shape feature which we can apply upgrade-shape to in unit-test cases.

@johnkerl johnkerl force-pushed the kerl/py-r-test-2 branch 16 times, most recently from 64e885c to fc64c3f Compare September 3, 2024 23:15
@johnkerl johnkerl changed the base branch from main to kerl/sdf-test-accessors September 3, 2024 23:30
@johnkerl johnkerl force-pushed the kerl/sdf-test-accessors branch from 5c03e17 to a40cf77 Compare September 3, 2024 23:32
@johnkerl johnkerl force-pushed the kerl/sdf-test-accessors branch from a40cf77 to 9ee2797 Compare September 3, 2024 23:42
@johnkerl johnkerl force-pushed the kerl/sdf-test-accessors branch from 9ee2797 to ee19025 Compare September 4, 2024 02:25
@johnkerl johnkerl force-pushed the kerl/py-r-test-2 branch 4 times, most recently from f467719 to bce3172 Compare September 4, 2024 13:33
@johnkerl johnkerl changed the base branch from kerl/sdf-test-accessors to kerl/feature-flag-temp September 4, 2024 13:34
@johnkerl johnkerl force-pushed the kerl/feature-flag-temp branch from aeb5141 to 8a883be Compare September 6, 2024 20:48
@johnkerl johnkerl force-pushed the kerl/py-r-test-2 branch 2 times, most recently from b0933d4 to 0f96d28 Compare September 7, 2024 02:30
@johnkerl johnkerl changed the base branch from kerl/feature-flag-temp to kerl/py-r-creation-paths September 7, 2024 03:13
@johnkerl johnkerl force-pushed the kerl/py-r-test-2 branch 4 times, most recently from ac08eff to 4bcbae5 Compare September 7, 2024 03:17
@johnkerl johnkerl changed the title [python/r] Iterative current-domain testing [temporary/WIP] [python/r] New-shape mutators resize and upgrade_shape Sep 7, 2024
@johnkerl johnkerl force-pushed the kerl/py-r-creation-paths branch from 0b5bddb to 192b63f Compare September 7, 2024 03:35
@johnkerl johnkerl force-pushed the kerl/py-r-creation-paths branch from 192b63f to a735be3 Compare September 7, 2024 03:58
@johnkerl johnkerl force-pushed the kerl/py-r-creation-paths branch from a735be3 to 407fa35 Compare September 7, 2024 04:16
Comment thread apis/python/src/tiledbsoma/soma_array.cc Outdated
Comment thread apis/r/R/SOMADenseNDArray.R
Comment thread apis/r/R/SOMASparseNDArray.R Outdated
Comment thread apis/r/R/SOMASparseNDArray.R Outdated
Comment thread apis/r/R/Init.R
Comment thread apis/r/tests/testthat/test-shape.r
@johnkerl
Copy link
Copy Markdown
Contributor Author

@mojaveazure ready for round two!

Comment thread apis/r/R/Init.R Outdated
Comment thread apis/r/R/SOMASparseNDArray.R Outdated
Comment thread apis/r/R/SOMASparseNDArray.R
@codecov
Copy link
Copy Markdown

codecov Bot commented Sep 11, 2024

Codecov Report

Attention: Patch coverage is 78.57143% with 6 lines in your changes missing coverage. Please review.

Project coverage is 89.79%. Comparing base (ff2eb5b) to head (68b9767).
Report is 7 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2950      +/-   ##
==========================================
+ Coverage   89.73%   89.79%   +0.06%     
==========================================
  Files          39       39              
  Lines        4080     4096      +16     
==========================================
+ Hits         3661     3678      +17     
+ Misses        419      418       -1     
Flag Coverage Δ
python 89.79% <78.57%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
python_api 89.79% <78.57%> (+0.06%) ⬆️
libtiledbsoma ∅ <ø> (∅)

@johnkerl
Copy link
Copy Markdown
Contributor Author

johnkerl commented Sep 11, 2024

@nguyenv and @mojaveazure ready for the next round! :)

Copy link
Copy Markdown
Member

@mojaveazure mojaveazure left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you! Please add a changelog in NEWS.md, bump the develop version in DESCRIPTION, and then 🚢

Copy link
Copy Markdown
Contributor

@nguyenv nguyenv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm but minor suggestion

Comment on lines +120 to +140
.def(
"resize",
[](SOMAArray& array, const std::vector<int64_t>& newshape) {
try {
array.resize(newshape);
} catch (const std::exception& e) {
throw TileDBSOMAError(e.what());
}
},
"newshape"_a)

.def(
"tiledbsoma_upgrade_shape",
[](SOMAArray& array, const std::vector<int64_t>& newshape) {
try {
array.upgrade_shape(newshape);
} catch (const std::exception& e) {
throw TileDBSOMAError(e.what());
}
},
"newshape"_a);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should do the rethrow in libtiledbsoma rather than pybind11. It'll simplify the code in here to be a one-liner again and the C++ API and R API would also consistently see TileDBSOMAError when erroring out from these methods.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish that worked reliably! We found on #2980 and #2963 that, unfortunately, it does not. This has to be a catch-and-rethrow right here at the pybind layer, unfortunately. :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants