-
Notifications
You must be signed in to change notification settings - Fork 10.3k
TSDB Commit should honor ordering between floats and native histograms #15177
Description
Proposal
Background
Spun out of #15142.
Related to #15112.
The current tsdb/head_append.go headAppender.Commit processes floats first, then integer native histograms and then float native histograms. This might not be the order of the samples actually.
There is a side effect in the memSeries.lastHistogram/memSeries.lastFloatHistogram handling as well, more details in #15142 .
Requirements
The desired solution would:
- honor the ordering,
- set lastHistogram/lastFloatHistogram in Commit() (possibly in WAL replay) only,
- turn float stale markers into histograms in Commit() instead of append.
Fallout
Does impact WAL/WBL replay, possibly remote write?
Implementation idea
The trivial idea would be to add one more slice that specifies the type of the next sample. We'd iterate the new array and take the next sample from the appropriate slice of a.samples , a.histograms or a.floatHistograms.
Optimize by not storing the type for every sample, rather remember how many samples are of the next type. For example for all floats this would just say that the type is float and the number is the same as len(a.samples).