Skip to content

OTEP: Autoinstrumentation system packages for DEB- and RPM-based Linux distributions#4793

Closed
mmanciop wants to merge 5 commits into
open-telemetry:mainfrom
mmanciop:otep-linux-system-packages
Closed

OTEP: Autoinstrumentation system packages for DEB- and RPM-based Linux distributions#4793
mmanciop wants to merge 5 commits into
open-telemetry:mainfrom
mmanciop:otep-linux-system-packages

Conversation

@mmanciop

Copy link
Copy Markdown

Changes

Add OTEP for system packages that deliver autoinstrumentations and the OTel Injector on Debian- and RedHat-based Linux distros.

@mmanciop
mmanciop requested review from a team as code owners December 10, 2025 05:49
@mmanciop mmanciop changed the title Start OTEP for autoinstrumentation system packages OTEP: Autoinstrumentation system packages for DEB- and RPM-based Linux distributions Dec 10, 2025
@mmanciop
mmanciop force-pushed the otep-linux-system-packages branch from 3a2adf6 to c9508b5 Compare December 10, 2025 05:54
@mmanciop
mmanciop force-pushed the otep-linux-system-packages branch from c9508b5 to 5d19ae3 Compare December 10, 2025 05:56

@atoulme atoulme left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mtwo mtwo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mmanciop thanks for writing this! One thing that I'd like to see is a plan that combines or rationalizes the injector's automatic discovery capabilities with the eBPF work going on in OBI. Have we put much thought into this yet?

@atoulme FYI

Comment thread oteps/4793-linux-system-packages.md Outdated

## Prior art and alternatives

The [Dash0 Operator](https://github.com/dash0hq/dash0-operator) has been using very similar mechanics (delivering an injector and autoinstrumentation files for various SDKs) on Kubernetes in production for more than year. The [OpenTelemetry Operator](https://github.com/open-telemetry/opentelemetry-operator/) also has injection mechanics processed runtime inside pods' containers, but the injection requires opt-in at the pod level via labels.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@atoulme might be worth adding the Splunk automatic configuration + discovery work here

Comment thread oteps/4793-linux-system-packages.md Outdated
@svrnm

svrnm commented Dec 16, 2025

Copy link
Copy Markdown
Member

Since this is a spec proposal, I am wondering if it would be possible to extend the "system packages for DEB- and RPM-based linux distributions" to APIs and SDKs, that would profit from that, as well. Mainly C++, but others maybe as well? And for those who this is not relevant/applicable because they have an eco-system for their SDK/API installation (nuget, npmjs, ...) this could be called out as "out of scope"

@mmanciop

mmanciop commented Dec 17, 2025

Copy link
Copy Markdown
Author

Since this is a spec proposal, I am wondering if it would be possible to extend the "system packages for DEB- and RPM-based linux distributions" to APIs and SDKs, that would profit from that, as well. Mainly C++, but others maybe as well? And for those who this is not relevant/applicable because they have an eco-system for their SDK/API installation (nuget, npmjs, ...) this could be called out as "out of scope"

I do not think that packaging SDKs and APIs as system packages for the languages listed in this proposal helps adoption. The language-specific packages do that work well. Distros may end up packaging OTel SDKs in the (IMO very unlikely) possibility that they become shared dependencies of other packages in main repositories (this is how Ubuntu ended up with a few DEB for Java and Node.js libraries).

C++ is a completely different beast entirely. C++ build systems are a fair bit outside my zone of comfort, but what I see is that, to this day, there's surprisingly little commonality among build setups for C++ projects, and several competing standards for C++ dependency package formats.

@mmanciop

mmanciop commented Dec 17, 2025

Copy link
Copy Markdown
Author

@mmanciop thanks for writing this! One thing that I'd like to see is a plan that combines or rationalizes the injector's automatic discovery capabilities with the eBPF work going on in OBI. Have we put much thought into this yet?

@atoulme FYI

I have not put much thought into this. However, I could imagine OBI being added as a package in the proposed package system to:

  1. Cover the programming languages that cannot be autoinjected (C, C++, Go, Rust)
  2. Provide alternative packages to the language-specific autoinstrumentation packages (e.g., Java auto instrumentation) to reflect the fact that the end user wishes to use OBI instead of the respective language-native SDK and autoinstrumentation.

I'd gladly talk it over with OBI folks, if they are interested :-)

@atoulme

atoulme commented Dec 19, 2025

Copy link
Copy Markdown
Contributor

@mmanciop thanks for writing this! One thing that I'd like to see is a plan that combines or rationalizes the injector's automatic discovery capabilities with the eBPF work going on in OBI. Have we put much thought into this yet?
@atoulme FYI

I have not put much thought into this. However, I could imagine OBI being added as a package in the proposed package system to:

  1. Cover the programming languages that cannot be autoinjected (C, C++, Go, Rust)
  2. Provide alternative packages to the language-specific autoinstrumentation packages (e.g., Java auto instrumentation) to reflect the fact that the end user wishes to use OBI instead of the respective language-native SDK and autoinstrumentation.

I'd gladly talk it over with OBI folks, if they are interested :-)

I agree with that approach. From a discussion at the injector SIG meeting with @rafaelroquetto I think there is interest. Rafael explained that OBI would not conflict with the injector because it detects if an OTLP exporter is loaded in the code, and deactivates in that case. I also work with @MrAlias who works on OBI and might have a take here.

@mmanciop

Copy link
Copy Markdown
Author

Since this is a spec proposal, I am wondering if it would be possible to extend the "system packages for DEB- and RPM-based linux distributions" to APIs and SDKs, that would profit from that, as well. Mainly C++, but others maybe as well? And for those who this is not relevant/applicable because they have an eco-system for their SDK/API installation (nuget, npmjs, ...) this could be called out as "out of scope"

Very out of scope IMO.

The OpenTelemetry system packages respect all the usual mechanics in terms of delivering updates via package managers:

* Easy to add a trusted package repository to the Linux system
* Dependencies are resolved transparently by the package management system: for, when you have already installed the `opentelemetry` package and later add a runtime to your Linux system, e.g., a Java Virtual Machine, that can be automatically injected, you will be recommended (opt-in) to also install the OpenTelemetry autoinstrumentation package for Java.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would this recommendation work?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest metadata in DEB. RPM unfortunately does not have an equivalent, and using Recommends instead myst install unwanted runtimes.

Autoinstrumentation packages delivering SDKs that support the [declarative configuration format](https://github.com/open-telemetry/opentelemetry-configuration) should integrate file-based configurations in their system package.

Injection of autoinstrumentations into applications is configurable by end users.
The OpenTelemetry Injector package offers, upon installation, a choice whether injecting applications is opt-in (default) or opt-out. In both cases, opting specific applications in or out is done via that application's process environment using a dedicated `OTEL_`-prefixed environment variable.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you imagining piggy-backing off of an existing env var like OTEL_SDK_ENABLED? Or introducing a new one, something like OTEL_INSTRUMENTATION_ENABLED?

Also, how do you see this interacting with the injector's include / exclude mechanism?


These system packages are designed to work together: the OpenTelemetry Injector knows where to look for SDKs and autoinstrumentations available on the system via their respective system packages, and injects only the applications running on matching runtimes.

Autoinstrumentation packages are modelled in terms of metadata and file-system content so that vendor-specific distribution can be used as opposed to the community autoinstrumentation packages provided by the OpenTelemetry SDKs ("upstream" packages). This requires the existence of runtime-specific metapackages, like `opentelemetry-java-autoinstrumentation`, which are `provided` (using the `.deb` terminology) by both the upstream packages and vendor packages; additionally, vendor packages `replace` (using the `.deb` terminology) the upstream packages.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To do this type of thing we'd need to introduce rails to standardize the publishing / organization of these autoinstrumentation packages. Probably a repo like opentelemetry-sdk-packages or something to that effect.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. There's a PROJECT file being written up ;-)

@mmanciop

mmanciop commented Feb 3, 2026

Copy link
Copy Markdown
Author

As agreed at OTel Unplugged '26, this OTEP is closed in favour of chartering the Packaging SIG :-)

@mmanciop mmanciop closed this Feb 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants