RUMM-2186: Write batch metadata#943
Conversation
| @WorkerThread | ||
| private fun writeData(byteArray: ByteArray): Boolean { | ||
| val file = fileOrchestrator.getWritableFile(byteArray.size) ?: return false | ||
| if (!checkEventSize(byteArray.size)) return false |
There was a problem hiding this comment.
Means that this event will be lost ? I wonder if this will create problems for SessionReplay records 🤔 , I guess they are not that big though.
There was a problem hiding this comment.
Yes, it will be lost. But this behavior is not different from the current one. Currently if item is too big we won't provide any file for writing it
| // This file may go away after rework, not adding tests for now (it is a simplified copy of | ||
| // BatchFileHandler) | ||
| @Suppress("unused") | ||
| internal class PlainFileHandler( |
There was a problem hiding this comment.
shouldn't be PlainSingleItemFileHandler ?
There was a problem hiding this comment.
Probably yes, but it will be a subject of RUMM-2235 anyway.
| internal val serializer: Serializer<T>, | ||
| internal val handler: FileHandler, | ||
| internal val internalLogger: Logger | ||
| internal val handler: ChunkedFileHandler, |
There was a problem hiding this comment.
shouldn't be a SingleItemFileHandler here ?
There was a problem hiding this comment.
Ideally, but it will be a subject of RUMM-2235 anyway. Currently the only implementation of ChunkedFileHandler is BatchFileHandler which will write data in TLV format adding meta as well, which are (both meta and TLV) not really needed for the use-cases of SingleItemDataWriter which are NdkUserInfoDataWriter and NdkNetworkInfoDataWriter (since we just want to store plain json), but these cases will work with BatchFileHandler as well.
| } | ||
|
|
||
| val file = orchestrator.getWritableFile(event.size) | ||
| if (!checkEventSize(event.size)) { |
There was a problem hiding this comment.
Isn't this already checked in the FileHandler ?
There was a problem hiding this comment.
It was moved from FileOrchestrator to the DataWriter, but DataWriter is not used by the ConsentAwareStorage right now (and probably won't, in fact in the writer callback we need only Serializer, so logically Storage replaces DataWriter), so adding this check here as well.
3 participants
What does this PR do?
This change adds support for reading/writing optional batch metadata (in a non-typed format, currently as
ByteArray).Batch metadata file will be written to the batch data file and will be associated with it by the name (metadata file will have the same name as batch file, just with
_metadatasuffix).This change also does the following:
FileOrchestratorto prevent the possibility of the switching to the next batch file if current batch size + event size exceeds the limit (we can do this, because the limit set in the SDK is still less than the actual limit of the intake(s) and the margin we have is still bigger than maximum event size). This makes us to keep batch metadata file and batch file in sync for the particularBatchWriter(otherwise we could have a metadata file from another batch). Responsibility of checking event size is now shifted to theWriterlayer.Review checklist (to be filled by reviewers)