Skip to content

Add synchronous mode for serverless environments#227

Merged
duncanista merged 3 commits intomasterfrom
jordan.gonzalez/synchronous-mode
Apr 23, 2026
Merged

Add synchronous mode for serverless environments#227
duncanista merged 3 commits intomasterfrom
jordan.gonzalez/synchronous-mode

Conversation

@duncanista
Copy link
Copy Markdown
Contributor

@duncanista duncanista commented Apr 14, 2026

Summary

  • Add StatsdConfig.SynchronousMode option that bypasses the async background worker and sends metrics directly on the calling thread
  • Extract IStatsSender interface to enable swapping async/sync implementations
  • Rename StatsBufferize to AsynchronousBufferizedSender for clarity
  • Suppress the telemetry background timer when synchronous mode is enabled
  • Forward exceptions to the configured error handler in synchronous mode; rethrow if no handler is set

Motivation

In AWS Lambda and similar serverless environments, the runtime freezes the sandbox immediately after the handler returns. The DogStatsD client processes metrics on a background thread via AsynchronousWorker, so buffered metrics can be lost when the freeze hits. The existing Flush() method has a hardcoded 3-second timeout and still relies on a background thread to drain the queue — making it fundamentally unreliable in serverless contexts.

SynchronousMode eliminates this class of bugs entirely: no background threads, no async queue, no race with the sandbox freeze. Metrics are routed and serialized on the calling thread, batched into efficient UDP packets via the existing BufferBuilder, and the user calls Flush() at the end of their handler to send any remaining partial buffer.

Usage

var dogstatsd = new DogStatsdService();
dogstatsd.Configure(new StatsdConfig
{
    StatsdServerName = "127.0.0.1",
    SynchronousMode = true,
});

// In Lambda handler:
dogstatsd.Counter("lambda.invocations", 1);
dogstatsd.Flush(); // all metrics sent before handler returns

Test plan

  • All existing tests pass with renamed types
  • 10 unit tests for SynchronousSender (pool behavior, send/flush, batching, auto-flush on full buffer, dispose triggers flush, thread safety, empty flush, exception forwarding to handler, exception rethrow when no handler)
  • 2 builder tests verifying sync mode wiring (skips async CreateAsynchronousBufferizedSender, passes synchronousMode to telemetry)
  • 10 integration tests with real UDP listener (counter, gauge, histogram, distribution, timer, set, tags, event, service check, dispose-flush)
  • Manual validation in a Lambda environment
Screenshot 2026-04-22 at 12 17 53 PM

In AWS Lambda and similar serverless environments, background threads
get frozen when the handler returns, causing buffered metrics to be
lost. This adds a SynchronousMode option that bypasses the async queue
and sends metrics directly on the calling thread.

Changes:
- Extract IStatsBufferize interface from StatsBufferize
- Add SynchronousSender that routes metrics synchronously
- Add StatsdConfig.SynchronousMode property (default false)
- Suppress telemetry background timer in sync mode
- Add unit tests, builder tests, and integration tests
@duncanista duncanista requested a review from a team as a code owner April 14, 2026 14:50
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a138a5f742

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/StatsdClient/Bufferize/SynchronousSender.cs Outdated
In the async path, AsynchronousWorker.Dequeue() catches exceptions from
Route/Flush and forwards them to optionalExceptionHandler. The sync
path was letting exceptions propagate directly into user code, which
could crash Lambda handlers on transport failures or serialization
errors. Wrap Send and Flush in try-catch to match async behavior.
Copy link
Copy Markdown
Contributor

@StephenWakely StephenWakely left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. I think this can definitely be useful! I just have some minor nits on the naming.

/// <summary>
/// IStatsBufferize defines the contract for sending stats to the pipeline.
/// </summary>
internal interface IStatsBufferize : IDisposable
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The naming of this interface is confusing. It doesn't really capture what it is trying to do. I think IStatsSender would make more sense.

Then perhaps SynchronousSender and AsynchronousBufferizedSender implementing them.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed as suggested: IStatsBufferizeIStatsSender, StatsBufferizeAsynchronousBufferizedSender. Also renamed the factory types for consistency (IStatsBufferizeFactoryIStatsSenderFactory, StatsBufferizeFactoryStatsSenderFactory).

}
catch (Exception e)
{
_optionalExceptionHandler?.Invoke(e);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it is a good idea to swallow the exceptions if a handler isn't specified.

I understand the purpose, given asynchronous does hide the exceptions, it does mean you can seamlessly swap between synchronous and asynchronous. But I imagine someone using a synchronous sender would expect the exceptions to be surfaced right away.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about it, to keep consistent with the async version, we should have the optionalExceptionHandler here - but if it isn't defined rethrow the exception rather than swallow it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated: when optionalExceptionHandler is null, exceptions are now rethrown instead of swallowed. When a handler is provided, the existing behavior is preserved (caught and forwarded to the handler). Note that through the normal DogStatsdService.Configure path, the handler is always set — it defaults to Debug.WriteLine when the caller does not provide one — so in practice exceptions are always caught and logged. The rethrow is a safety net for direct construction without a handler. Added a test to verify the rethrow behavior.

StephenWakely
StephenWakely previously approved these changes Apr 21, 2026
Copy link
Copy Markdown
Contributor

@StephenWakely StephenWakely left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

Rename IStatsBufferize -> IStatsSender, StatsBufferize -> AsynchronousBufferizedSender,
and factory types for consistency per reviewer suggestion. Update SynchronousSender to
rethrow exceptions when no handler is configured instead of silently swallowing them.
Copy link
Copy Markdown
Contributor

@StephenWakely StephenWakely left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@duncanista duncanista merged commit 478de5c into master Apr 23, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants