Skip to content

TSDB Commit should honor ordering between floats and native histograms #15177

@krajorama

Description

@krajorama

Proposal

Background

Spun out of #15142.
Related to #15112.

The current tsdb/head_append.go headAppender.Commit processes floats first, then integer native histograms and then float native histograms. This might not be the order of the samples actually.

There is a side effect in the memSeries.lastHistogram/memSeries.lastFloatHistogram handling as well, more details in #15142 .

Requirements

The desired solution would:

  • honor the ordering,
  • set lastHistogram/lastFloatHistogram in Commit() (possibly in WAL replay) only,
  • turn float stale markers into histograms in Commit() instead of append.

Fallout

Does impact WAL/WBL replay, possibly remote write?

Implementation idea

The trivial idea would be to add one more slice that specifies the type of the next sample. We'd iterate the new array and take the next sample from the appropriate slice of a.samples , a.histograms or a.floatHistograms.
Optimize by not storing the type for every sample, rather remember how many samples are of the next type. For example for all floats this would just say that the type is float and the number is the same as len(a.samples).

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions