feat(core): add data_id with current_data_id() SQL function#5917
feat(core): add data_id with current_data_id() SQL function#5917bluestreak01 merged 28 commits intomasterfrom
Conversation
…nctionality through SQL.
RaphDal
changed the title
…mprove open logic
eugenels
left a comment
eugenels
left a comment
There was a problem hiding this comment.
Hey Raphael, thanks for the PR!
I think we can simplify the design here. Ideally, this could just be a static utility function: take a file path as input, return an existing or new ID as output. Since the ID is meant to be generated once and never updated, we don’t need to keep the file descriptor or memory map around for the lifetime of CairoEngine.
You can also utilize MemoryCMARWImpl directly - it encapsulates all the file and mmap logic internally. All you need to do is tune the sizes appropriately. For your case, setting extendSegmentSize = 32 and size = -1 will allocate a 32-byte file.
Also, I don't think we need to modify SqlExecutionContext - we already have access to getCairoEngine(), which should be sufficient.
Let me know what you think!
… error handling tests
…hod in DataIDUtils
…ifiers and update related functionality
...ava/io/questdb/test/griffin/engine/functions/catalogue/CurrentDataIDFunctionFactoryTest.java
Show resolved
Hide resolved
amunra
left a comment
amunra
left a comment
There was a problem hiding this comment.
Minor code cleanup improvements needed.
[PR Coverage check]😍 pass : 34 / 37 (91.89%) file detail
|
5 participants
This PR introduces a data identifier to distinguish separate clusters and avoid conflicts on replication and restore.
Currently, users can query it through the
current_data_id()function in SQL. The long term goal is to store this data id in replications and backups so that on restore a check can be made against replications and restores configuration and abort in case of conflicts.From an implementation standpoint, this id is stored in the root of the database under the
.data_idfilename and consists of a 128bit UUID.