Skip to content

[r] Use libtiledbsoma for array handles #3061

@johnkerl

Description

@johnkerl

Currently we're retaining a tiledb-r handle at open:

private$.tiledb_array <- tiledb::tiledb_array_open(self$object, type = mode)

And we use a temporary open-use-close at every single call to libtiledbsoma -- here is just one of many examples:

void set_metadata(std::string& uri, std::string& key, SEXP valuesxp, std::string& type,
bool is_array, Rcpp::XPtr<somactx_wrap_t> ctxxp,
Rcpp::Nullable<Rcpp::DatetimeVector> tsvec = R_NilValue) {
// shared pointer to SOMAContext from external pointer wrapper
std::shared_ptr<tdbs::SOMAContext> sctx = ctxxp->ctxptr;
// SOMA Object unique pointer (aka soup)
auto soup = getObjectUniquePointer(is_array, OpenMode::write, uri, sctx, tsvec);
if (type == "character") {
const tiledb_datatype_t value_type = TILEDB_STRING_UTF8;
std::string value = Rcpp::as<std::string>(valuesxp);
spdl::debug("[set_metadata] key {} value {} is_array {} type {}", key, value, is_array, type);
soup->set_metadata(key, value_type, value.length(), (void*) value.c_str(), true);
} else if (type == "integer64") {
const tiledb_datatype_t value_type = TILEDB_INT64;
double dv = Rcpp::as<double>(valuesxp);
int64_t value = Rcpp::fromInteger64(dv);
spdl::debug("[set_metadata] key {} value {} is_array {} type {}", key, value, is_array, type);
soup->set_metadata(key, value_type, 1, (void*) &value, true);
} else {
Rcpp::stop("Unsupported type '%s'", type);
}
soup->close();
}

On the one hand this might seem lower-pri: the redundant opens are a perf hit but they work.

But as discussed on #3060 we must do this in order to remove the tiledb-r dependency.

See also #3053 which @nguyenv is working on -- this is a case where we do currently require a second open for array reads.

From #3059:

Already done for groups

#2406; [sc-55685].

Steps:

  • Status quo is
    • There is a private$.tiledb_array with connection to self$object
      • This is set on construction/open, and closed on close
    • The libtiledbsoma handles are currently ephemeral
      • Every read / write / get_metadata etc which is libtiledbsoma-capable currently takes a uri and a ctx and does open/op/close
  • Needing to be done:

To check:

  • Currently the array set_metadata does not error out when the array is opened for read -- this is a bug.
    • This should be auto-fixed by this issue, but, it needs to be tested.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions