@jack-berg I mentioned a while back that the documentation for this type of declarative config addition was minimal, so would love for you to review the implementation here.

I'm struggling to visualize the UX for this.

From what I can tell:

• You include this dynamic control jar on your class path.
• If declarative config is used TelemetryPolicyDeclarativeCustomizerProvider is invoked, which looks for the presence of top level .telemetry_policy/development.
• If not present, do nothing.
• If present, try to initialize:
  • Parse PolicyInitConfig from the node
  • Modify the OpenTelemetryConfiguration model to set the tracer provider sampler to be telemetry_policy/development, and some other properties set (sources, resource_attributes, otel.resource.attributes - not sure what these do what its a code smell).
• TelemetryPolicyComponentProvider sees the sampler set to telemetry_policy/development, and returns a sampler with a dynamic implementation wired up to the policy

If that's about right, then it seems convoluted. Maybe necessarily so based on the hooks available today, but if so, maybe the answer is to go and fix the hooks so this flows more cleanly.

In my head, a user should opt into dynamic control with a config file something like:

file_format: "1.0"
tracer_provider:
  sampler: ..
meter_provider: ...
logger_provider: ...
telemetry_policy/development: ...

When the .telemetry_policy/development is present, the telemetry policy machinery initializes, and ultimately gets a reference to the OpenTelemetrySdk instance returned by declarative config. From there, it can listen for changes to the policy (however those happen) and call update methods on the OpenTelemetrySdk instance.

I.e.

• if a policy changes that results in a new sampler config, call OpenTelemetrySdk.getSdkTracerProvider().setSampler(newSampler)
• if a policy changes that results loggers being turned off or on, call OpenTelemetrySdk.getSdkLoggerProvider().setLoggerConfigurator(newLoggerConfigurator)
• etc

Caveat: I'm not currently doing any dynamic config tasks. Just trying to help and make sure that we're not trying to solve this problem strictly through the tools we already have. We can change / add tools if needed.

I'm struggling to visualize the UX for this.

The README has the details

some other properties set (sources, resource_attributes, otel.resource.attributes - not sure what these do what its a code smell).

Some components need specific resource items at initialization. In the current particular case an OpAMP client needs to know both the service name and the environment

If that's about right, then it seems convoluted. Maybe necessarily so based on the hooks available today, but if so, maybe the answer is to go and fix the hooks so this flows more cleanly.

There are two different considerations

1. Getting this extension working now with the current hooks
2. Making the hooks better to make the implementation nicer

Please keep these separate. I want this extension functioning then we can evolve it. (Note it is already fully functioning for autoconfiguration, this particular PR enables it to function in the same way for declarative config).

In my head, a user should opt into dynamic control with a config file something like:

file_format: "1.0"
tracer_provider:
  sampler: ..
meter_provider: ...
logger_provider: ...
telemetry_policy/development: ...

This is exactly how it works with this PR.

When the .telemetry_policy/development is present, the telemetry policy machinery initializes, and ultimately gets a reference to the OpenTelemetrySdk instance returned by declarative config. From there, it can listen for changes to the policy (however those happen) and call update methods on the OpenTelemetrySdk instance.

I.e.

• if a policy changes that results in a new sampler config, call OpenTelemetrySdk.getSdkTracerProvider().setSampler(newSampler)
• if a policy changes that results loggers being turned off or on, call OpenTelemetrySdk.getSdkLoggerProvider().setLoggerConfigurator(newLoggerConfigurator)
• etc

These don't currently work. Sampler can't be set there, and even if you added it, it's currently final at initialization. Logging is much worse, you need to install your preferred logging framework at exactly the right point, not too early (or you hit cyclic dependencies) nor too late (it just won't install correctly), and definitely not dynamically currently

I agree this is the way it should work. We can open a separate issue on changing the SDK to make it work, but that's really extensive changes even for these two features, and there are many more to consider

What I'm hoping for here in review terms is whether the declarative config implementation is the correct way for what needs to be done here, ie

• parse the telemetry_policy/development to initialize the policy pipeline
• install the required sampler needed to support dynamically changing sampling rate

Please keep these separate.

👍

Sampler can't be set there,

Just because we haven't made it mutable yet. I don't think there is any fundamental limitation from making it mutable like the logger / tracer / meter configurators.

Logging is much worse,

I'm not following. Updating the otel log SDK with the set of loggers that should be enabled / disabled is an independent problem from installing / initializing your preferred log framework.

What I'm hoping for here in review terms is whether the declarative config implementation is the correct way for what needs to be done here, ie

parse the telemetry_policy/development to initialize the policy pipeline
install the required sampler needed to support dynamically changing sampling rate

I won't claim there is a correct way at this point, but it does look like you've managed to make it work despite subpar hooks.

See recent comments for thoughts. None are blocking. Just things to consider since I'm not an owner of this component.

Just because we haven't made it mutable yet. I don't think there is any fundamental limitation from making it mutable like the logger / tracer / meter configurators.

I'll open an issue in the SDK with changes suggested

Logging is much worse,

I'm not following. Updating the otel log SDK with the set of loggers that should be enabled / disabled is an independent problem from installing / initializing your preferred log framework.

In the Elastic distribution we install log4j as the preferred logging framework. When you do that you also have to initialize it, otherwise weird things happen. If you do that too early, you get a class circularity error (and in fact when too early is changes with different SDK versions because the class circularity is with MethodHandles and is painful). If you leave it too late, log4j doesn't correctly install into the SDK and you get a sort of partial logging.

• ✅ login: jackshirazi / name: Jack Shirazi (4097e36, 70b2a8d, 7aad3f2, 8ea0e2e, c4cff36, e8e82f4, e8fe336, ef658c8)
• ❌ - login: @cursoragent / name: Cursor. The commit (4097e36, 70b2a8d) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please visit our EasyCLA portal and chat with our support bot.

moved change set to #2967 (unless someone knows how to authorize cursor as co-author)