Add topology-driven log generation#145
Merged
Merged
Conversation
Add a logs: section to service and operation definitions, replacing the
need for hardcoded ERROR/WARN derivations as the only log output:
- Severity levels TRACE through FATAL per the OTel log data model
- Body templates with {key} interpolation from log attributes, span
attributes, and service/operation built-ins
- Conditional emission (error, success, slow) and per-span probability
- Timing anchors (span start or end) with an optional delay offset
- Structured attributes using the existing attribute generators
- Trace correlation: records carry the producing span's trace/span IDs
Services that define no logs keep the derived error/slow behaviour;
defining any logs on a service replaces the derived logs for it.
Closes #112
There was a problem hiding this comment.
Pull request overview
Adds topology-driven log generation to the synth engine, allowing services/operations to define log templates in the YAML DSL (severity, body templates, conditions, probability sampling, timing anchors/delay, and generated attributes), while preserving existing derived error/slow logs for services that don’t define any logs:. Also adds trace/span correlation to emitted log records via SpanContext.
Changes:
- Extend the topology DSL and resolved topology model to support
logs:at service and operation scope, plus validation and example documentation. - Implement a new
LogObserverthat emits topology-defined log records (with interpolation/conditions/probability/timing) and falls back to derived logs when no templates exist for a service. - Propagate span
SpanContextthroughSpanInfoso emitted logs include trace/span IDs.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/synth/topology.go | Adds resolved LogDefinition and resolves YAML LogConfig into templates on services/operations. |
| pkg/synth/topology_test.go | Tests that topology log configs resolve correctly and invalid delay is rejected. |
| pkg/synth/observer.go | Extends SpanInfo to carry trace.SpanContext for correlation. |
| pkg/synth/logs.go | Replaces derived-only observer with topology-template emission + derived fallback; adds interpolation, conditions, sampling, and correlation. |
| pkg/synth/logs_test.go | Adds extensive tests for topology log behavior and trace correlation; updates existing derived-log tests for new constructor. |
| pkg/synth/engine.go | Passes span.SpanContext() into SpanInfo for observers. |
| pkg/synth/engine_test.go | Updates to new NewLogObserver signature (now returns (*LogObserver, error)). |
| pkg/synth/emit.go | Passes SpanContext into SpanInfo for observer notifications in emit paths. |
| pkg/synth/config.go | Adds DSL types/constants + validation for logs: at service/operation scope. |
| pkg/synth/config_test.go | Adds log load/validation tests, but also removes a large set of existing metric override/interval tests. |
| docs/examples/topology-driven-logs.yaml | Adds runnable example demonstrating topology-driven logs. |
| cmd/motel/README.md | Documents the new logs DSL, interpolation rules, conditions, defaults, and correlation. |
| cmd/motel/main.go | Wires topology and RNG into NewLogObserver creation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Restore the opRef assignment in the ValidateConfig operation loop and the metric override, walk/bounds, and interval tests that the merge resolution dropped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes issue 112 (topology-driven log and event generation).
Adds a
logs:section to service and operation definitions, following the same design as topology-driven metrics: service-level entries are evaluated for every span in the service, operation-level entries only for that operation.DSL
Features
{key}placeholders resolve against the record's own attributes, then span attributes, thenservice.name/operation.namebuilt-ins; unresolved placeholders stay literalerror,success,slow(slow uses--slow-thresholdand never fires when the flag is unset); omitted = every spanprobability(0-1)at: start|endanchor plusdelayoffset, applied to the record timestamp so it works with synthetic/offset timeSpanInfonow carries the producing span'sSpanContext, and records are emitted with it on the context, so every log record (including the derived ones) carries trace/span IDsAnswers to the issue's design questions
logs:(at either level) replaces the derived logs for that service; other services keep them. The README documents how to recreate the derived behaviour explicitly.probability.Span events (issue 86) remain a separate concept; this only covers standalone log records.
Validation and docs
ValidateConfigchecks severity, body, condition, probability range, timing anchor, delay, and attribute generators with the usual contextual error messagescmd/motel/README.mdand a runnable example atdocs/examples/topology-driven-logs.yamlVerified end-to-end with
motel run --stdout --signals traces,logs --slow-threshold 100mson the new example: topology logs interpolate span attributes, conditions and probability behave as configured, the no-logs service still emits derived WARN/ERROR records, and all records carry trace/span IDs.https://claude.ai/code/session_01AoVS3SJVCebSzegzL5aGuX
Generated by Claude Code