Skip to content

perf(forwarder): compile multiline regex patterns once at module level#1078

Merged
LorisFriedel merged 2 commits into
masterfrom
loris.friedel/perf-module-level-regex
Mar 17, 2026
Merged

perf(forwarder): compile multiline regex patterns once at module level#1078
LorisFriedel merged 2 commits into
masterfrom
loris.friedel/perf-module-level-regex

Conversation

@LorisFriedel

Copy link
Copy Markdown
Member

Summary

  • DD_MULTILINE_LOG_REGEX_PATTERN was compiled into 2 regex objects on every S3EventHandler instantiation
  • For SQS batches with ~10 records, that is 20 redundant re.compile calls per invocation
  • Move the two compilations to module-level constants (_MULTILINE_REGEX_START_PATTERN, _MULTILINE_REGEX_PATTERN) and assign the pre-compiled references in __init__

Test plan

  • Existing test_s3_handler.py tests pass (tests set instance attributes directly, so they are unaffected)
  • Verify multiline log regex still works end-to-end via integration tests

🤖 Generated with Claude Code

DD_MULTILINE_LOG_REGEX_PATTERN was compiled into 2 regex objects on
every S3EventHandler instantiation. For SQS batches with ~10 records,
that is 20 redundant re.compile calls per invocation.

Move the two compilations to module-level constants and assign the
pre-compiled references in __init__.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@LorisFriedel
LorisFriedel requested a review from a team as a code owner March 16, 2026 10:29
@github-actions github-actions Bot added the aws label Mar 16, 2026
CI's black version requires 1 blank line (not 2) between the import
block and module-level variable assignments. Two blank lines are
reserved for class and function definitions.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@ViBiOh ViBiOh self-assigned this Mar 17, 2026
@LorisFriedel
LorisFriedel merged commit ae92da5 into master Mar 17, 2026
10 checks passed
@LorisFriedel
LorisFriedel deleted the loris.friedel/perf-module-level-regex branch March 17, 2026 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants