Investigation so far shows that Python dense-array write calls C++ set_array_data as well as set_dim_points, while R dense-array write calls only the former and not the latter.

cc @mojaveazure and your observation the other day that the dense-array writer doesn't handle coords. Furthermore, as presently coded, it can only write the full core domain of the array. And now that with the new-shape feature flag enabled, the core domain (soma maxdomain) is huge and the core current domain (soma domain) is small.

Here is a repro that fails even with the new-shape feature flag disabled, proving that this bug is older than the new-shape mod:

#!/usr/bin/env Rscript

library(tiledbsoma)
library(arrow)

uri <- "data-dnda-r"
element_type = arrow::float32()
shape = c(10, 20)

cat("CR-DNDA URI     ", uri, "\n")
cat("CR-DNDA SHAPE   ", shape, "\n")

if (dir.exists(uri)) unlink(uri, recursive=TRUE)

dnda <- tiledbsoma::SOMADenseNDArrayCreate(uri, type=element_type, shape=shape)

cat("WR-DNDA URI", uri, "\n")

dnda$write(matrix(1:12, nrow=3,ncol=4))

dnda$close()

CR-DNDA URI      data-dnda-r 
CR-DNDA SHAPE    10 20 
WR-DNDA URI data-dnda-r 
Error: WriterBase: Buffer sizes check failed; Invalid number of cells given for attribute 'soma_data' (12 != 200)
Execution halted

Also fails with

...
m <- matrix(1:12, nrow=3,ncol=4)
coords <- list( 4:6, 7:10)
dnda$write(m, coords)
...

Same error message.

The typeguard-related failures are new and weird. I created #3245 to check.

CI run from commit 1 failed: https://github.com/single-cell-data/TileDB-SOMA/actions/runs/11542968229/job/32126790602?pr=3245

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.92%. Comparing base (50c5d71) to head (0e6493a).
Report is 2 commits behind head on main.

@@            Coverage Diff             @@
##             main    #3244      +/-   ##
==========================================
+ Coverage   83.79%   83.92%   +0.12%     
==========================================
  Files          51       51              
  Lines        5560     5566       +6     
==========================================
+ Hits         4659     4671      +12     
+ Misses        901      895       -6     

@nguyenv this is ready for review, and has green CI for this repo (i.e. with core 2.26).

There is still an issue or two remaining for core dev, which I have installed on one laptop to mimic @jdblischak 's nightlies. So, there will be at least one more PR. Meanwhile everything here on this PR is already a strict improvment.

Moving back to draft, likely to abandon later on.

On discussion with @nguyenv :

• The main context of this PR is:
  • For dense arrays only
  • If any dim's subarray range isn't specified, core uses the core domain
  • With new-shape feature-flag enabled, and with core 2.27 where dense arrays can have current domain (they couldn't in 2.26), we are indeed using data-sized core current domain and huge core domain
  • Put those together and we get these problems
  • Note there are a few other issues on this PR, and a few not yet included in this PR, other than that -- and those should be split out into separate PRs
• On this PR I've ben finding callsites, for write and for read, for Python and R, and touching up the ranges which end up being core subarrays on each dim slot
  • This is tedious and error-prone
• What we should do instead is:
  • In one place -- in libtiledbsoma, in ManagedQuery, before submit_query
  • If any dim's range has not been set up to this point, and if the array has a current domain, simply set the ranges on that dim to be the current domain's (lo, hi)
  • We know (for a fact) that core will go ahead and use the too-big domain anyway so this is safe

Closing in favor of #3263 and #3268, and at least one more PR to come after those.