Repository navigation
serverboot: give the ateoms a LoggerProvider over the relay - #1857
Merged
Jeff Luo (JeffLuoo) merged 2 commits intoSep 24, 2026
Merged
Conversation
LoggingOptions gains ExporterConn and RelayCapable, the same two fields TracingOptions has, so a component can send log records over a connection it owns and mark its resource with the path taken. Both ateoms now initialize logging beside tracing and metrics, passing the relay connection. The exporter still defaults to none, so no ateom emits a record yet; this only puts the provider in place for the actor events that will move into the ateom. newLoggerProvider is split out so a test can read the resource off an emitted record, since the SDK provider does not expose it. The observability guide names the rule a consumer applies to lifecycle records: only ateapi's resource is authoritative, because the relay admits only ateom resources.
Tim Bai (baizhenyu)
requested review from
Jeff Luo (JeffLuoo),
Da Huang (git286) and
Krisztian F (krisztianfekete)
September 24, 2026 15:22
Krisztian F (krisztianfekete)
previously approved these changes
Sep 24, 2026
Krisztian F (krisztianfekete)
left a comment
Krisztian F (krisztianfekete)
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, added some nit comments!
| **What this stream won't tell you.** It only writes when something changes. So an actor that has been sitting in the same state since before your logs roll over has no record, and no state. Ask the control plane what state something is in right now. Use this stream to see how it got there and when. Records can also go missing, like any other log, and a gap looks the same as an actor that just sat still. If you want to count activations, use the router's access log instead. | ||
|
|
||
| `Actor crashed` is the exception, and carries the same two keys with `ate.actor.state="crashed"`. It is written once per committed transition into `ACTOR_STATE_CRASHED`, beside the [`ate.actor.crashes`](#the-metric-registry) increment and under the same already-crashed guard, so the two can never disagree about how many crashes happened. A consumer deriving state therefore selects on `ate.actor.state`, not on the message: | ||
| `Actor crashed` is the exception, and carries the same two keys with `ate.actor.state="crashed"`. It is written once per committed transition into `ACTOR_STATE_CRASHED`, beside the [`ate.actor.crashes`](#the-metric-registry) increment and under the same already-crashed guard, so the two can never disagree about how many crashes happened. A consumer deriving state therefore selects on `ate.actor.state` under ateapi's resource (`service.name=ateapi`), not on the message. The resource is what makes the record authoritative: worker pods export through [the ateom relay](#the-ateom-otlp-relay) when they have it, and the relay admits only ateom resources, so a record carrying these keys under any other resource is not a state change. |
Collaborator
There was a problem hiding this comment.
The relay only gates the relay path, and a worker pod that falls back to direct export can still send service.name=ateapi to the collector, so please state the rule as holding once that path is closed (a #809 follow-up) rather than as something the relay already guarantees.
Collaborator
Author
There was a problem hiding this comment.
Agreed. Reworded in c1e619d. The rule now holds once the direct path is closed. Until then the consumer applies it.
| #### The same records over OTLP | ||
|
|
||
| Both records also go out as OTLP log events, so a collector reads them without knowing substrate's stdout envelope. Set `OTEL_LOGS_EXPORTER=otlp` to turn it on; unset means `none`, which is what every environment but kind uses today. Only ateapi has a LoggerProvider today; [the ateom relay](#the-ateom-otlp-relay) carries logs, traces, and metrics, so an ateom exports log records the same way once it has one. | ||
| Both records also go out as OTLP log events, so a collector reads them without knowing substrate's stdout envelope. Set `OTEL_LOGS_EXPORTER=otlp` to turn it on; unset means `none`, which is what every environment but kind uses today. ateapi is the only emitter today. The ateoms have a LoggerProvider on the same switch and export through [the ateom relay](#the-ateom-otlp-relay), which carries logs, traces, and metrics. |
Collaborator
There was a problem hiding this comment.
atecontroller doesn't pass OTEL_LOGS_EXPORTER to worker pods, so setting it in ate-otel-config doesn't turn on the ateoms; can we please document it here as well?
Collaborator
Author
There was a problem hiding this comment.
Added in c1e619d. The controller does not pass the switch, so kind turns on ateapi only.
| } | ||
| // Nil when the exporter is none. | ||
| if lp != nil { | ||
| defer serverboot.ShutdownProvider("LoggerProvider", lp.Shutdown) |
Collaborator
There was a problem hiding this comment.
Nice!
Bowei Du (bowei)
added
kind/feature
The relay gates only the relay path. A worker pod that falls back to direct export can still send any resource, so the rule is one the consumer applies until that path is closed. Also note that the controller does not pass the logs exporter switch to worker pods, so the kind ConfigMap reaches ateapi only.
Jeff Luo (JeffLuoo)
approved these changes
Sep 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 2 of #1748, plus the consumer rule from step 1b.
LoggingOptionsgainsExporterConnandRelayCapable, mirroringTracingOptions. Both ateoms initialize logging after metrics with the relay connection and defer a nil-guarded shutdown.OTEL_LOGS_EXPORTERstill defaults tonone, so no ateom emits a record; this is the provider the actor events land on when emission moves.A relay-capable log resource carries
ate.otlp.relaylike traces and metrics do.newLoggerProvideris split fromInitLoggingso the test asserts that on an emitted record's resource, the wayTestMeterProviderRelayAttributedoes for metrics.docs/observability.mdstates the consumer rule that replaced the relay denylist dropped in #1800: a lifecycle record is authoritative only under ateapi's resource, because the relay admits only ateom resources.Follow-up for the emission step, not here: the controller does not inject
OTEL_LOGS_EXPORTERinto worker pods, so no environment exercises log-over-relay yet.