Skip to content

POC, DO NOT PIC: OTLP to Appender#12152

Closed
krajorama wants to merge 4 commits intomainfrom
krajo/otlp-to-appender
Closed

POC, DO NOT PIC: OTLP to Appender#12152
krajorama wants to merge 4 commits intomainfrom
krajo/otlp-to-appender

Conversation

@krajorama
Copy link
Copy Markdown
Contributor

@krajorama krajorama commented Jul 21, 2025

Adopt prometheus/prometheus#16855 .

Will have to be updated to solution in prometheus/prometheus#16951

Remove copy of Prometheus OTLP translator code from Mimir as it is now storage independent in (mimir-)Prometheus.

Add new CombinedAppender implementation that uses the mimirpb.PreAllocTimeseries as target.

@krajorama krajorama requested review from aknuds1 and ywwg July 21, 2025 14:19
krajorama added a commit that referenced this pull request Jul 24, 2025
krajorama added a commit that referenced this pull request Jul 24, 2025
krajorama added a commit that referenced this pull request Jul 24, 2025
krajorama added a commit that referenced this pull request Jul 25, 2025
Change the test to exercise labels much more.

Use 2000 series instead of 1.
Use 1 sample per series instead of 1000.
Use more realistic number of labels per series.
Make 10% of series use exponential histograms and let those have 10
exemplars per series instead of 1 per series.
Add exemplars to the first sample for each series.

Related to #12152

---------

Signed-off-by: György Krajcsovits <[email protected]>
Co-authored-by: Copilot <[email protected]>
krajorama added a commit that referenced this pull request Jul 28, 2025
Change the test to exercise labels much more.

Use 2000 series instead of 1.
Use 1 sample per series instead of 1000.
Use more realistic number of labels per series.
Make 10% of series use exponential histograms and let those have 10
exemplars per series instead of 1 per series.
Add exemplars to the first sample for each series.

Related to #12152

---------

Signed-off-by: György Krajcsovits <[email protected]>
Co-authored-by: Copilot <[email protected]>
# Conflicts:
#	pkg/distributor/otel_test.go
Copy link
Copy Markdown
Contributor

@aknuds1 aknuds1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments.

Comment thread pkg/distributor/otel.go
Comment thread pkg/distributor/otel.go
}

func newOTLPMimirConverter() *otlpMimirConverter {
func newOTLPMimirConverter(appender *otlpappender.CombinedAppender) *otlpMimirConverter {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit] Is there any benefit to take the otlpappender.CombinedAppender argument? Can't it just be instantiated in this function, instead of leaving it to the caller?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, I haven't spent a lot of time on it, not sure why it's so complicated currently.

krajorama added a commit that referenced this pull request Aug 4, 2025
Prepare for the rewrite of the OTLP translator upstream in
prometheus/prometheus#16951
by adding an end-to-end test to check for regressions.

Related to #12152.

Also switch from raw JSON in the test to building the payload
in code and marshaling to protobuf.

Signed-off-by: György Krajcsovits <[email protected]>
Signed-off-by: György Krajcsovits <[email protected]>

# Conflicts:
#	go.mod
#	go.sum
#	vendor/github.com/prometheus/prometheus/storage/remote/otlptranslator/prometheusremotewrite/helper.go
#	vendor/github.com/prometheus/prometheus/storage/remote/otlptranslator/prometheusremotewrite/number_data_points.go
#	vendor/modules.txt
Signed-off-by: György Krajcsovits <[email protected]>
Signed-off-by: György Krajcsovits <[email protected]>

# Conflicts:
#	pkg/distributor/otel.go
#	pkg/distributor/otel_test.go
#	pkg/distributor/push_test.go
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama krajorama force-pushed the krajo/otlp-to-appender branch from e463606 to 0f12324 Compare August 13, 2025 14:34
@krajorama krajorama changed the base branch from r351 to main August 13, 2025 14:35
@krajorama
Copy link
Copy Markdown
Contributor Author

outdated, new PR: #12652

@krajorama krajorama closed this Sep 8, 2025
krajorama added a commit that referenced this pull request Sep 9, 2025
Prepare for the rewrite of the OTLP translator upstream in
prometheus/prometheus#16951
by adding an end-to-end test to check for regressions.

Related to #12152.

Also switch from raw JSON in the test to building the payload
in code and marshaling to protobuf.

Signed-off-by: György Krajcsovits <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants