Skip to content

Add scenario overrides for topology-driven logs#156

Merged
andrewh merged 2 commits into
mainfrom
claude/jolly-lovelace-7lcmf0
Jun 11, 2026
Merged

Add scenario overrides for topology-driven logs#156
andrewh merged 2 commits into
mainfrom
claude/jolly-lovelace-7lcmf0

Conversation

@andrewh

@andrewh andrewh commented Jun 11, 2026

Copy link
Copy Markdown
Owner

Closes #155. Follow-up to PR 145 (topology-driven log generation), mirroring the metric override pattern from OverrideConfig.Metrics.

What

Scenarios can now modify log output during their activation window via a logs: block under override:

scenarios:
  - name: db-degradation
    at: 2m
    duration: 5m
    override:
      postgres.query:
        error_rate: 15%
        logs:
          add:
            - severity: ERROR
              body: "connection pool exhausted"
              condition: error
              attributes:
                error.type:
                  value: PoolExhaustedError
          disable: true   # optional: mute base logs at this scope
  • add: defines extra log records emitted only while the scenario window is active. Entries are full LogConfig definitions: severity, body templates, conditions (error/success/slow), probability, timing anchors (at/delay), and attribute generators all work.
  • disable: true mutes the base logs at the override's scope for the duration of the window — both topology log templates and the derived error/slow fallback logs. Combined with add:, this gives replacement semantics.
  • Both override scopes work, consistent with metric overrides: service.operation refs scope to one operation; bare service names scope to every span of the service.

Design decisions (per the issue's open questions)

  1. Identity for modifying existing definitions: deferred, as suggested. This first iteration supports add: and disable: true only; no name/id field was added to LogConfig. Log-storm simulation is achievable today with disable: true plus an add: entry that copies the base log with a higher probability.
  2. Derived-vs-topology switching: adding logs to a service that previously had none switches it to topology logs for the window (suppressing the derived error/slow fallback), and disable mutes the derived logs too — so the suppression rule is now scenario-aware.

Implementation

  • LogOverrideConfig (add, disable) on OverrideConfig, validated at config load with the usual contextual messages (each add entry goes through validateLogConfig; an empty logs: block is rejected).
  • BuildScenarios resolves added entries to LogDefinitions via the existing resolveLogs (its error-context parameter was generalised to support scenario contexts).
  • ResolveOverrides merges across overlapping scenarios: adds accumulate in priority order, disable from any active scenario wins.
  • LogObserver implements the existing OverrideObserver interface; the engine's notifyOverrides path needed no changes. Added definitions are pre-built into templates on scenario transitions so the per-span path only does map lookups under an RWMutex, matching MetricObserver.

Testing

  • Unit tests for validation, scenario building/merging, and observer behaviour (operation/service scoping, conditions, interpolation, disable semantics, derived-log interaction, override clearing).
  • Verified end-to-end with the updated docs/examples/topology-driven-logs.yaml: the added ERROR log appears only inside the window on error spans, the gateway's base logs go quiet during it, and behaviour reverts when the window closes.
  • make test and make lint pass.

https://claude.ai/code/session_01L12XgTDZiZMBQU6cMoFLpC


Generated by Claude Code

Scenarios can now modify log output during their activation window via a
logs: block under override, mirroring the metric override shape. add:
defines extra log records (full LogConfig: severity, body templates,
conditions, probability, timing anchors, attributes) emitted only while
the scenario is active; disable: true mutes the base logs (topology
templates and derived error/slow logs) at the override's scope.

Overrides work at both scopes: service.operation refs and bare service
names, consistent with metric overrides. Added definitions are validated
at config load with contextual error messages, resolved in
BuildScenarios, merged across overlapping scenarios (adds accumulate,
disable from any active scenario wins), and applied by LogObserver via
the existing OverrideObserver notification path.

Closes #155

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds scenario-time-window overrides for topology-driven logs, mirroring the existing scenario override model for metrics, so incident scenarios can add extra log templates and/or mute base logs for a given service or operation scope.

Changes:

  • Generalize resolveLogs error-context handling to support scenario override contexts.
  • Introduce logs: scenario override support (add, disable) across config validation, scenario building/merging, and runtime observer application.
  • Extend unit tests and docs example(s) to cover scenario log override behavior.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
pkg/synth/topology.go Generalizes resolveLogs API to accept an explicit error-context string.
pkg/synth/scenario.go Carries parsed/merged log overrides (AddLogs, DisableLogs) through scenario resolution.
pkg/synth/scenario_test.go Adds unit tests for parsing and merging scenario log overrides.
pkg/synth/logs.go Implements scenario-aware log emission (mute base logs + emit added templates) via SetOverrides.
pkg/synth/logs_test.go Adds LogObserver tests for add/disable semantics and derived-log interactions.
pkg/synth/config.go Adds LogOverrideConfig and validates scenario logs: overrides at config load time.
pkg/synth/config_test.go Adds validation tests for scenario logs: overrides at both scopes.
docs/examples/topology-driven-logs.yaml Demonstrates logs.add and logs.disable scenario override usage.
docs/examples/README.md Documents the updated topology-driven logs example.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/synth/logs.go Outdated
Comment thread pkg/synth/logs_test.go
Address review: SetOverrides silently skipped newLogTemplate errors,
which could mask a future drift between validLogSeverity and
severityByName. Derive validLogSeverity from severityByName so the
validation set cannot diverge from the severities the observer can
emit, and make newLogTemplate infallible: severity is rejected at
config load, and a hand-constructed unknown severity maps to
SeverityUndefined while preserving the severity text rather than
dropping the record.

Also add coverage for operation-scoped disable muting derived logs.
@andrewh
andrewh merged commit f21af76 into main Jun 11, 2026
2 checks passed
@andrewh
andrewh deleted the claude/jolly-lovelace-7lcmf0 branch June 11, 2026 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scenario overrides for topology-driven logs

3 participants