Skip to content

perf(forwarder): parse event message JSON once in split()#1080

Merged
LorisFriedel merged 2 commits into
masterfrom
loris.friedel/perf-single-json-parse
Mar 17, 2026
Merged

perf(forwarder): parse event message JSON once in split()#1080
LorisFriedel merged 2 commits into
masterfrom
loris.friedel/perf-single-json-parse

Conversation

@LorisFriedel

@LorisFriedel LorisFriedel commented Mar 16, 2026

Copy link
Copy Markdown
Member

Summary

  • extract_metric() and extract_trace_payload() both called json.loads(event["message"]) independently
  • For plain log events (the majority), both fail with exceptions — 2x parse + 2x exception overhead per event
  • Parse once in split() and pass the pre-parsed dict directly to the extractors
  • extract_metric(parsed, event) validates and enriches a pre-parsed dict
  • is_trace(parsed) is a pure predicate — the trace payload dict is built in split() since it only needs the raw message and tags
  • Events that fail JSON parsing are classified as logs immediately, skipping both extractors

Test plan

  • test_splitting.py tests pass (12 tests covering metric validation, trace detection, and edge cases)
  • Verify log/metric/trace splitting works end-to-end via integration tests

🤖 Generated with Claude Code

extract_metric() and extract_trace_payload() both called
json.loads(event["message"]) independently. For plain log events (the
majority), both fail with exceptions — 2x parse + 2x exception
overhead per event.

Parse once in split() via _try_parse_message() and pass the result to
both extractors through a parsed_message parameter. A _PARSE_FAILED
sentinel avoids conflating parse failure with a missing argument.

Short-circuit in split(): when a metric is found, skip the trace check
entirely. This is both more efficient and avoids a shared-dict mutation
issue where extract_metric appends to the parsed dict's "t" list.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@LorisFriedel
LorisFriedel requested a review from a team as a code owner March 16, 2026 10:29
@github-actions github-actions Bot added the aws label Mar 16, 2026
Comment thread aws/logs_monitoring/steps/splitting.py Outdated
@ViBiOh ViBiOh self-assigned this Mar 16, 2026
Address review feedback: instead of threading a parsed_message param
through extract_metric and extract_trace_payload, parse once in split()
and pass the parsed dict directly.

- split() handles json.loads and classifies parse failures as logs
- extract_metric(parsed, event) validates and enriches a pre-parsed dict
- is_trace(parsed) is a pure predicate — the trace payload dict is
  built in split() since it only needs the raw message and tags
- Removes _PARSE_FAILED sentinel and _try_parse_message helper

Tests updated to match the new signatures with better coverage of
actual validation logic.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@LorisFriedel
LorisFriedel requested a review from ViBiOh March 16, 2026 21:31
@LorisFriedel
LorisFriedel merged commit 7694bbe into master Mar 17, 2026
10 checks passed
@LorisFriedel
LorisFriedel deleted the loris.friedel/perf-single-json-parse branch March 17, 2026 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants