Skip to content

Update Samples.AWS.Lambda and tests#3359

Merged
andrewlock merged 3 commits into
masterfrom
andrew/serverless-tests
Oct 19, 2022
Merged

Update Samples.AWS.Lambda and tests#3359
andrewlock merged 3 commits into
masterfrom
andrew/serverless-tests

Conversation

@andrewlock

@andrewlock andrewlock commented Oct 14, 2022

Copy link
Copy Markdown
Member

Summary of changes

  • Refactor tests, and add some explainers about how it all works
  • Added custom instrumentation to the serverless tests
  • Don't fail with "unknown address" in all tests
  • Include the "implicit" spans in the snapshots

Reason for change

I need to work on the serverless implementation, and I was finding the serverless tests quite confusing as there's a lot of containers and a lot of back and forth. The changes in this PR add documentation, move the "dummy" lambda extension into the integration test (instead of using separate containers), make the behaviour more "realistic", and add manual instrumentation in for good measure. This PR updates the snapshots as appropriate.

Implementation details

From the docs (added in this PR), this is now what the implementation looks like. The main changes I made were to add the dummy API (previously it was always sending requests to an API that doesn't exist), add the manual span, and to make the "context container" a part of the test fixture.

sequenceDiagram
    participant xunit
    participant Samples.Aws.Lambda
    participant MockLambdaExtension
    participant MockTracerAgent
    participant Handler
    participant Tracer
    participant Dummy API
    autonumber
    
    xunit->>+Samples.Aws.Lambda: StartProcessAndWaitForExit()
    
    loop For every handler container
        Samples.Aws.Lambda->>+Handler : Post()
        Handler->>+MockLambdaExtension: /lambda/start-invocation()
        MockLambdaExtension-->>-Handler: Context/null
        Handler->>+Tracer : Start dummy Span
        Handler->>+Tracer : Start manual span()
        Handler->>+Dummy API: GET() (instrumented)
        activate Tracer
        Dummy API-->>-Handler: 200 OK
        deactivate Tracer
        Tracer-->>-Handler: Stop manual span()
        Handler->>+MockLambdaExtension: /lambda/end-invocation()
        MockLambdaExtension-->>-Handler: 
        Handler-->>-Samples.Aws.Lambda : 200 OK
        Tracer-->-Handler : Dummy Span ends
    end
    
    Samples.Aws.Lambda-->>xunit: exit 0
    deactivate Samples.Aws.Lambda
        
    Note over xunit,MockTracerAgent: Integration Test container
    Note over Handler,Tracer: Handler container
    Note over Dummy API: Dummy API container
Loading

I also added extra tests for handlers that explicitly generate an error, and for several different type of handler arguments.

Test coverage

Coverage is basically the same, should be a little better. The snapshot file looks a bit like the following, where there's lot of independent traces. These new snapshots highlight a couple of problematic cases where we're not instrumenting the handlers correctly

ui perfetto dev_ (5)

Other details

The lambda instrumentation automatically creates a span when a handler method is called. The traces in the snapshots highlight the (new) test cases I added where we're not correctly creating the handler trace because we're failing to instrument in the LambdaHandler:

  • The generic dictionary parameter case.
  • Nested class parameter
  • Nested struct parameter

image
image

I'll add support for these in a subsequent PR

@andrewlock andrewlock added area:tests unit tests, integration tests area:test-apps apps used to test integrations area:serverless labels Oct 14, 2022
@andrewlock
andrewlock requested review from a team as code owners October 14, 2022 17:15
@github-actions

This comment was marked as outdated.

@andrewlock

This comment has been minimized.

@andrewlock

This comment has been minimized.

- Don't auto-throw on every span, call a real API instead
- Add manual spans in each handler
- Create a `MockLambdaExtension` (based on the `MockTracerAgent`) to give easier control (and testing) of context responses (replaces the "context" containers)
- Add additional parameter types (currently unsupported)
- Update the snapshots to include the "implicit" spans created by the extension endpoints
@andrewlock
andrewlock force-pushed the andrew/serverless-tests branch from b0b7875 to 063ac4b Compare October 18, 2022 11:29
@andrewlock
andrewlock requested a review from a team as a code owner October 18, 2022 11:29
@github-actions

Copy link
Copy Markdown
Contributor

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer.
The diff is simplistic, so please check some files anyway while we improve it.

1 occurrences of :

+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_3,
+    SpanId: Id_4,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_5,
+    SpanId: Id_6,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_7,
+    SpanId: Id_8,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_9,
+    SpanId: Id_10,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_11,
+    SpanId: Id_12,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_13,
+    SpanId: Id_14,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_15,
+    SpanId: Id_16,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_17,
+    SpanId: Id_18,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_19,
+    SpanId: Id_20,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_21,
+    SpanId: Id_22,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_23,
+    SpanId: Id_24,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_25,
+    SpanId: Id_26,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_27,
+    SpanId: Id_28,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_29,
+    SpanId: Id_30,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_31,
+    SpanId: Id_32,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_33,
+    SpanId: Id_34,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_35,
+    SpanId: Id_36,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_37,
+    SpanId: Id_38,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Error: 1,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_39,
+    SpanId: Id_40,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Error: 1,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_41,
+    SpanId: Id_42,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Error: 1,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_43,
+    SpanId: Id_44,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Error: 1,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_45,
+    SpanId: Id_46,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Error: 1,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_47,
+    SpanId: Id_48,
+    Name: lambda.invocation,
+    Resource: /lambda/end-invocation,
+    Service: LambdaExtension,
+    Error: 1,
+    Tags: {
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_1,
+    SpanId: Id_49,
+    Name: manual.HandlerNoParamSync,
+    Resource: manual.HandlerNoParamSync,
+    Service: Bootstrap,
+    ParentId: Id_2,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_1,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_3,
+    SpanId: Id_50,
+    Name: manual.HandlerOneParamSync,
+    Resource: manual.HandlerOneParamSync,
+    Service: Bootstrap,
+    ParentId: Id_4,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_2,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_5,
+    SpanId: Id_51,
+    Name: manual.HandlerTwoParamsSync,
+    Resource: manual.HandlerTwoParamsSync,
+    Service: Bootstrap,
+    ParentId: Id_6,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_3,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_7,
+    SpanId: Id_52,
+    Name: manual.HandlerNoParamAsync,
+    Resource: manual.HandlerNoParamAsync,
+    Service: Bootstrap,
+    ParentId: Id_8,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_4,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_9,
+    SpanId: Id_53,
+    Name: manual.HandlerOneParamAsync,
+    Resource: manual.HandlerOneParamAsync,
+    Service: Bootstrap,
+    ParentId: Id_10,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_5,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_11,
+    SpanId: Id_54,
+    Name: manual.HandlerTwoParamsAsync,
+    Resource: manual.HandlerTwoParamsAsync,
+    Service: Bootstrap,
+    ParentId: Id_12,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_6,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_13,
+    SpanId: Id_55,
+    Name: manual.HandlerNoParamVoid,
+    Resource: manual.HandlerNoParamVoid,
+    Service: Bootstrap,
+    ParentId: Id_14,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_7,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_15,
+    SpanId: Id_56,
+    Name: manual.HandlerOneParamVoid,
+    Resource: manual.HandlerOneParamVoid,
+    Service: Bootstrap,
+    ParentId: Id_16,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_8,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_17,
+    SpanId: Id_57,
+    Name: manual.HandlerTwoParamsVoid,
+    Resource: manual.HandlerTwoParamsVoid,
+    Service: Bootstrap,
+    ParentId: Id_18,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_9,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_19,
+    SpanId: Id_58,
+    Name: manual.HandlerNoParamSyncWithContext,
+    Resource: manual.HandlerNoParamSyncWithContext,
+    Service: Bootstrap,
+    ParentId: Id_20,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_10
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_21,
+    SpanId: Id_59,
+    Name: manual.HandlerOneParamSyncWithContext,
+    Resource: manual.HandlerOneParamSyncWithContext,
+    Service: Bootstrap,
+    ParentId: Id_22,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_11
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_23,
+    SpanId: Id_60,
+    Name: manual.HandlerTwoParamsSyncWithContext,
+    Resource: manual.HandlerTwoParamsSyncWithContext,
+    Service: Bootstrap,
+    ParentId: Id_24,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_12
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_25,
+    SpanId: Id_61,
+    Name: manual.BaseHandlerNoParamSync,
+    Resource: manual.BaseHandlerNoParamSync,
+    Service: Bootstrap,
+    ParentId: Id_26,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_13,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_27,
+    SpanId: Id_62,
+    Name: manual.BaseHandlerTwoParamsSync,
+    Resource: manual.BaseHandlerTwoParamsSync,
+    Service: Bootstrap,
+    ParentId: Id_28,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_14,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_29,
+    SpanId: Id_63,
+    Name: manual.BaseHandlerOneParamSyncWithContext,
+    Resource: manual.BaseHandlerOneParamSyncWithContext,
+    Service: Bootstrap,
+    ParentId: Id_30,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_15
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_31,
+    SpanId: Id_64,
+    Name: manual.BaseHandlerOneParamAsync,
+    Resource: manual.BaseHandlerOneParamAsync,
+    Service: Bootstrap,
+    ParentId: Id_32,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_16,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_33,
+    SpanId: Id_65,
+    Name: manual.BaseHandlerTwoParamsVoid,
+    Resource: manual.BaseHandlerTwoParamsVoid,
+    Service: Bootstrap,
+    ParentId: Id_34,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_17,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_35,
+    SpanId: Id_66,
+    Name: manual.HandlerStructParam,
+    Resource: manual.HandlerStructParam,
+    Service: Bootstrap,
+    ParentId: Id_36,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_18,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_37,
+    SpanId: Id_67,
+    Name: manual.ThrowingHandler,
+    Resource: manual.ThrowingHandler,
+    Service: Bootstrap,
+    ParentId: Id_38,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_19,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_39,
+    SpanId: Id_68,
+    Name: manual.ThrowingHandlerAsync,
+    Resource: manual.ThrowingHandlerAsync,
+    Service: Bootstrap,
+    ParentId: Id_40,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_20,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_41,
+    SpanId: Id_69,
+    Name: manual.ThrowingHandlerAsyncTask,
+    Resource: manual.ThrowingHandlerAsyncTask,
+    Service: Bootstrap,
+    ParentId: Id_42,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_21,
+      _dd.p.dm: -0
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_43,
+    SpanId: Id_70,
+    Name: manual.ThrowingHandler,
+    Resource: manual.ThrowingHandler,
+    Service: Bootstrap,
+    ParentId: Id_44,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_22
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_45,
+    SpanId: Id_71,
+    Name: manual.ThrowingHandlerAsync,
+    Resource: manual.ThrowingHandlerAsync,
+    Service: Bootstrap,
+    ParentId: Id_46,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_23
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_47,
+    SpanId: Id_72,
+    Name: manual.ThrowingHandlerAsyncTask,
+    Resource: manual.ThrowingHandlerAsyncTask,
+    Service: Bootstrap,
+    ParentId: Id_48,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_24
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 1.0
+    }
+  },
+  {
+    TraceId: Id_1,
+    SpanId: Id_73,
+    Name: http.request,
+    Resource: GET serverless-dummy-api:9005/function/HandlerNoParamSync,
+    Service: Bootstrap-http-client,
+    Type: http,
+    ParentId: Id_49,
+    Tags: {
+      component: WebRequest,
+      http.method: GET,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerNoParamSync,
+      runtime-id: Guid_1,
+      span.kind: client
+    },
+    Metrics: {
+      _dd.top_level: 1.0
+    }
+  },
+  {
+    TraceId: Id_3,
+    SpanId: Id_74,
+    Name: http.request,
+    Resource: GET serverless-dummy-api:9005/function/HandlerOneParamSync,
+    Service: Bootstrap-http-client,
+    Type: http,
+    ParentId: Id_50,
+    Tags: {
+      component: WebRequest,
+      http.method: GET,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerOneParamSync,
+      runtime-id: Guid_2,
+      span.kind: client
+    },
+    Metrics: {
+      _dd.top_level: 1.0
+    }
+  },
+  {
+    TraceId: Id_5,
+    SpanId: Id_75,
+    Name: http.request,
+    Resource: GET serverless-dummy-api:9005/function/HandlerTwoParamsSync,
+    Service: Bootstrap-http-client,
+    Type: http,
+    ParentId: Id_51,
+    Tags: {
+      component: WebRequest,
+      http.method: GET,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerTwoParamsSync,
+      runtime-id: Guid_3,
+      span.kind: client
+    },
+    Metrics: {
+      _dd.top_level: 1.0
+    }
+  },
+  {
+    TraceId: Id_7,
+    SpanId: Id_76,
+    Name: http.request,
+    Resource: GET serverless-dummy-api:9005/function/HandlerNoParamAsync,
+    Service: Bootstrap-http-client,
+    Type: http,
+    ParentId: Id_52,
+    Tags: {
+      component: WebRequest,
+      http.method: GET,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerNoParamAsync,
+      runtime-id: Guid_4,
+      span.kind: client
+    },
+    Metrics: {
+      _dd.top_level: 1.0
+    }
+  },
+  {
+    TraceId: Id_9,
+    SpanId: Id_77,
+    Name: http.request,
+    Resource: GET serverless-dummy-api:9005/function/HandlerOneParamAsync,
+    Service: Bootstrap-http-client,
+    Type: http,
+    ParentId: Id_53,
+    Tags: {
+      component: WebRequest,
+      http.method: GET,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerOneParamAsync,
+      runtime-id: Guid_5,
+      span.kind: client
+    },
+    Metrics: {
+      _dd.top_level: 1.0
+    }
+  },
+  {
+    TraceId: Id_11,
+    SpanId: Id_78,
+    Name: http.request,
+    Resource: GET serverless-dummy-api:9005/function/HandlerTwoParamsAsync,
+    Service: Bootstrap-http-client,
+    Type: http,
+    ParentId: Id_54,
+    Tags: {
+      component: WebRequest,
+      http.method: GET,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerTwoParamsAsync,
+      runtime-id: Guid_6,
+      span.kind: client
+    },
+    Metrics: {
+      _dd.top_level: 1.0
+    }
+  },
+  {
+    TraceId: Id_13,
+    SpanId: Id_79,
+    Name: http.request,
+    Resource: GET serverless-dummy-api:9005/function/HandlerNoParamVoid,
+    Service: Bootstrap-http-client,
+    Type: http,
+    ParentId: Id_55,
+    Tags: {
+      component: WebRequest,
+      http.method: GET,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerNoParamVoid,
+      runtime-id: Guid_7,
+      span.kind: client
+    },
+    Metrics: {
+      _dd.top_level: 1.0
+    }
+  },
+  {
+    TraceId: Id_15,
+    SpanId: Id_80,
+    Name: http.request,
+    Resource: GET serverless-dummy-api:9005/function/HandlerOneParamVoid,
+    Service: Bootstrap-http-client,
+    Type: http,
+    ParentId: Id_56,
+    Tags: {
+      component: WebRequest,
+      http.method: GET,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerOneParamVoid,
+      runtime-id: Guid_8,
+      span.kind: client
+    },
+    Metrics: {
+      _dd.top_level: 1.0
+    }
+  },
+  {
+    TraceId: Id_17,
+    SpanId: Id_81,
+    Name: http.request,
+    Resource: GET serverless-dummy-api:9005/function/HandlerTwoParamsVoid,
+    Service: Bootstrap-http-client,
+    Type: http,
+    ParentId: Id_57,
+    Tags: {
+      component: WebRequest,
+      http.method: GET,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerTwoParamsVoid,
+      runtime-id: Guid_9,
+      span.kind: client
+    },
+    Metrics: {
+      _dd.top_level: 1.0
+    }
+  },
+  {
+    TraceId: Id_19,
+    SpanId: Id_82,
[...]
-    Resource: GET localhost/function/BaseHandlerNoParamSync,
+    Resource: GET serverless-dummy-api:9005/function/HandlerNoParamSyncWithContext,
[...]
-    ParentId: Id_3,
-    Error: 1,
+    ParentId: Id_58,
[...]
-      error.msg: Cannot assign requested address Cannot assign requested address,
-      error.stack:
[...]
-      error.type: System.Net.WebException,
[...]
-      http.url: http://localhost/function/BaseHandlerNoParamSync,
-      runtime-id: Guid_1,
-      span.kind: client,
-      _dd.p.dm: -0
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerNoParamSyncWithContext,
+      runtime-id: Guid_10,
+      span.kind: client
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_4,
-    SpanId: Id_5,
+    TraceId: Id_21,
+    SpanId: Id_83,
[...]
-    Resource: GET localhost/function/BaseHandlerOneParamAsync,
+    Resource: GET serverless-dummy-api:9005/function/HandlerOneParamSyncWithContext,
[...]
-    ParentId: Id_6,
-    Error: 1,
+    ParentId: Id_59,
[...]
-      error.msg: Cannot assign requested address Cannot assign requested address,
-      error.stack:
[...]
-      error.type: System.Net.WebException,
[...]
-      http.url: http://localhost/function/BaseHandlerOneParamAsync,
-      runtime-id: Guid_2,
-      span.kind: client,
-      _dd.p.dm: -0
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerOneParamSyncWithContext,
+      runtime-id: Guid_11,
+      span.kind: client
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_7,
-    SpanId: Id_8,
+    TraceId: Id_23,
+    SpanId: Id_84,
[...]
-    Resource: GET localhost/function/BaseHandlerOneParamSyncWithContext,
+    Resource: GET serverless-dummy-api:9005/function/HandlerTwoParamsSyncWithContext,
[...]
-    ParentId: Id_9,
-    Error: 1,
+    ParentId: Id_60,
[...]
-      error.msg: Cannot assign requested address Cannot assign requested address,
-      error.stack:
[...]
-      error.type: System.Net.WebException,
[...]
-      http.url: http://localhost/function/BaseHandlerOneParamSyncWithContext,
-      runtime-id: Guid_3,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerTwoParamsSyncWithContext,
+      runtime-id: Guid_12,
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_10,
-    SpanId: Id_11,
+    TraceId: Id_25,
+    SpanId: Id_85,
[...]
-    Resource: GET localhost/function/BaseHandlerTwoParamsSync,
+    Resource: GET serverless-dummy-api:9005/function/BaseHandlerNoParamSync,
[...]
-    ParentId: Id_12,
-    Error: 1,
+    ParentId: Id_61,
[...]
-      error.msg: Cannot assign requested address Cannot assign requested address,
-      error.stack:
[...]
-      error.type: System.Net.WebException,
[...]
-      http.url: http://localhost/function/BaseHandlerTwoParamsSync,
-      runtime-id: Guid_4,
-      span.kind: client,
-      _dd.p.dm: -0
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/BaseHandlerNoParamSync,
+      runtime-id: Guid_13,
+      span.kind: client
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_13,
-    SpanId: Id_14,
+    TraceId: Id_27,
+    SpanId: Id_86,
[...]
-    Resource: GET localhost/function/BaseHandlerTwoParamsVoid,
+    Resource: GET serverless-dummy-api:9005/function/BaseHandlerTwoParamsSync,
[...]
-    ParentId: Id_15,
-    Error: 1,
+    ParentId: Id_62,
[...]
-      error.msg: Cannot assign requested address Cannot assign requested address,
-      error.stack:
[...]
-      error.type: System.Net.WebException,
[...]
-      http.url: http://localhost/function/BaseHandlerTwoParamsVoid,
-      runtime-id: Guid_5,
-      span.kind: client,
-      _dd.p.dm: -0
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/BaseHandlerTwoParamsSync,
+      runtime-id: Guid_14,
+      span.kind: client
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_16,
-    SpanId: Id_17,
+    TraceId: Id_29,
+    SpanId: Id_87,
[...]
-    Resource: GET localhost/function/HandlerNoParamAsync,
+    Resource: GET serverless-dummy-api:9005/function/BaseHandlerOneParamSyncWithContext,
[...]
-    ParentId: Id_18,
-    Error: 1,
+    ParentId: Id_63,
[...]
-      error.msg: Cannot assign requested address Cannot assign requested address,
-      error.stack:
[...]
-      error.type: System.Net.WebException,
[...]
-      http.url: http://localhost/function/HandlerNoParamAsync,
-      runtime-id: Guid_6,
-      span.kind: client,
-      _dd.p.dm: -0
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/BaseHandlerOneParamSyncWithContext,
+      runtime-id: Guid_15,
+      span.kind: client
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_19,
-    SpanId: Id_20,
+    TraceId: Id_31,
+    SpanId: Id_88,
[...]
-    Resource: GET localhost/function/HandlerNoParamSync,
+    Resource: GET serverless-dummy-api:9005/function/BaseHandlerOneParamAsync,
[...]
-    ParentId: Id_21,
-    Error: 1,
+    ParentId: Id_64,
[...]
-      error.msg: Cannot assign requested address Cannot assign requested address,
-      error.stack:
[...]
-      error.type: System.Net.WebException,
[...]
-      http.url: http://localhost/function/HandlerNoParamSync,
-      runtime-id: Guid_7,
-      span.kind: client,
-      _dd.p.dm: -0
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/BaseHandlerOneParamAsync,
+      runtime-id: Guid_16,
+      span.kind: client
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_7,
-    SpanId: Id_22,
+    TraceId: Id_33,
+    SpanId: Id_89,
[...]
-    Resource: GET localhost/function/HandlerNoParamSyncWithContext,
+    Resource: GET serverless-dummy-api:9005/function/BaseHandlerTwoParamsVoid,
[...]
-    ParentId: Id_23,
-    Error: 1,
+    ParentId: Id_65,
[...]
-      error.msg: Cannot assign requested address Cannot assign requested address,
-      error.stack:
[...]
-      error.type: System.Net.WebException,
[...]
-      http.url: http://localhost/function/HandlerNoParamSyncWithContext,
-      runtime-id: Guid_8,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/BaseHandlerTwoParamsVoid,
+      runtime-id: Guid_17,
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_24,
-    SpanId: Id_25,
+    TraceId: Id_35,
+    SpanId: Id_90,
[...]
-    Resource: GET localhost/function/HandlerNoParamVoid,
+    Resource: GET serverless-dummy-api:9005/function/HandlerStructParam,
[...]
-    ParentId: Id_26,
-    Error: 1,
+    ParentId: Id_66,
[...]
-      error.msg: Cannot assign requested address Cannot assign requested address,
-      error.stack:
[...]
-      error.type: System.Net.WebException,
[...]
-      http.url: http://localhost/function/HandlerNoParamVoid,
-      runtime-id: Guid_9,
-      span.kind: client,
-      _dd.p.dm: -0
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerStructParam,
+      runtime-id: Guid_18,
+      span.kind: client
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_27,
-    SpanId: Id_28,
+    TraceId: Id_37,
+    SpanId: Id_91,
[...]
-    Resource: GET localhost/function/HandlerOneParamAsync,
+    Resource: GET localhost/function/ThrowingHandler,
[...]
-    ParentId: Id_29,
+    ParentId: Id_67,

1 occurrences of :

-      http.url: http://localhost/function/HandlerOneParamAsync,
-      runtime-id: Guid_10,
-      span.kind: client,
-      _dd.p.dm: -0
+      http.url: http://localhost/function/ThrowingHandler,
+      runtime-id: Guid_19,
+      span.kind: client
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_30,
-    SpanId: Id_31,
+    TraceId: Id_39,
+    SpanId: Id_92,
[...]
-    Resource: GET localhost/function/HandlerOneParamSync,
+    Resource: GET localhost/function/ThrowingHandlerAsync,
[...]
-    ParentId: Id_32,
+    ParentId: Id_68,

1 occurrences of :

-      http.url: http://localhost/function/HandlerOneParamSync,
-      runtime-id: Guid_11,
-      span.kind: client,
-      _dd.p.dm: -0
+      http.url: http://localhost/function/ThrowingHandlerAsync,
+      runtime-id: Guid_20,
+      span.kind: client
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_7,
-    SpanId: Id_33,
+    TraceId: Id_41,
+    SpanId: Id_93,
[...]
-    Resource: GET localhost/function/HandlerOneParamSyncWithContext,
+    Resource: GET localhost/function/ThrowingHandlerAsyncTask,
[...]
-    ParentId: Id_34,
+    ParentId: Id_69,

1 occurrences of :

-      http.url: http://localhost/function/HandlerOneParamSyncWithContext,
-      runtime-id: Guid_12,
+      http.url: http://localhost/function/ThrowingHandlerAsyncTask,
+      runtime-id: Guid_21,
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_35,
-    SpanId: Id_36,
+    TraceId: Id_43,
+    SpanId: Id_94,
[...]
-    Resource: GET localhost/function/HandlerOneParamVoid,
+    Resource: GET localhost/function/ThrowingHandler,
[...]
-    ParentId: Id_37,
+    ParentId: Id_70,

1 occurrences of :

-      http.url: http://localhost/function/HandlerOneParamVoid,
-      runtime-id: Guid_13,
-      span.kind: client,
-      _dd.p.dm: -0
+      http.url: http://localhost/function/ThrowingHandler,
+      runtime-id: Guid_22,
+      span.kind: client
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_38,
-    SpanId: Id_39,
+    TraceId: Id_45,
+    SpanId: Id_95,
[...]
-    Resource: GET localhost/function/HandlerTwoParamsAsync,
+    Resource: GET localhost/function/ThrowingHandlerAsync,
[...]
-    ParentId: Id_40,
+    ParentId: Id_71,

1 occurrences of :

-      http.url: http://localhost/function/HandlerTwoParamsAsync,
-      runtime-id: Guid_14,
-      span.kind: client,
-      _dd.p.dm: -0
+      http.url: http://localhost/function/ThrowingHandlerAsync,
+      runtime-id: Guid_23,
+      span.kind: client
[...]
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
+      _dd.top_level: 1.0
[...]
-    TraceId: Id_41,
-    SpanId: Id_42,
+    TraceId: Id_47,
+    SpanId: Id_96,
[...]
-    Resource: GET localhost/function/HandlerTwoParamsSync,
+    Resource: GET localhost/function/ThrowingHandlerAsyncTask,
[...]
-    ParentId: Id_43,
+    ParentId: Id_72,

1 occurrences of :

-      http.url: http://localhost/function/HandlerTwoParamsSync,
-      runtime-id: Guid_15,
-      span.kind: client,
+      http.url: http://localhost/function/ThrowingHandlerAsyncTask,
+      runtime-id: Guid_24,
+      span.kind: client
+    },
+    Metrics: {
+      _dd.top_level: 1.0
+    }
+  },
+  {
+    TraceId: Id_97,
+    SpanId: Id_98,
+    Name: manual.HandlerGenericDictionaryParam,
+    Resource: manual.HandlerGenericDictionaryParam,
+    Service: Bootstrap,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_25,
[...]
+      _dd.agent_psr: 1.0,
[...]
-    TraceId: Id_7,
-    SpanId: Id_44,
+    TraceId: Id_97,
+    SpanId: Id_99,
[...]
-    Resource: GET localhost/function/HandlerTwoParamsSyncWithContext,
+    Resource: GET serverless-dummy-api:9005/function/HandlerGenericDictionaryParam,
[...]
-    ParentId: Id_45,
-    Error: 1,
+    ParentId: Id_98,
[...]
-      error.msg: Cannot assign requested address Cannot assign requested address,
-      error.stack:
[...]
-      error.type: System.Net.WebException,
[...]
-      http.url: http://localhost/function/HandlerTwoParamsSyncWithContext,
-      runtime-id: Guid_16,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerGenericDictionaryParam,
+      runtime-id: Guid_25,
[...]
+    Metrics: {
+      _dd.top_level: 1.0
+    }
+  },
+  {
+    TraceId: Id_100,
+    SpanId: Id_101,
+    Name: manual.HandlerNestedClassParam,
+    Resource: manual.HandlerNestedClassParam,
+    Service: Bootstrap,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_26,
+      _dd.p.dm: -0
+    },
[...]
+      _dd.agent_psr: 1.0,
[...]
-    TraceId: Id_46,
-    SpanId: Id_47,
+    TraceId: Id_100,
+    SpanId: Id_102,
[...]
-    Resource: GET localhost/function/HandlerTwoParamsVoid,
+    Resource: GET serverless-dummy-api:9005/function/HandlerNestedClassParam,
[...]
-    ParentId: Id_48,
-    Error: 1,
+    ParentId: Id_101,
[...]
-      error.msg: Cannot assign requested address Cannot assign requested address,
-      error.stack:
[...]
-      error.type: System.Net.WebException,
[...]
-      http.url: http://localhost/function/HandlerTwoParamsVoid,
-      runtime-id: Guid_17,
-      span.kind: client,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerNestedClassParam,
+      runtime-id: Guid_26,
+      span.kind: client
+    },
+    Metrics: {
+      _dd.top_level: 1.0
+    }
+  },
+  {
+    TraceId: Id_103,
+    SpanId: Id_104,
+    Name: manual.HandlerNestedStructParam,
+    Resource: manual.HandlerNestedStructParam,
+    Service: Bootstrap,
+    Tags: {
+      language: dotnet,
+      runtime-id: Guid_27,
[...]
+      _dd.agent_psr: 1.0,
[...]
+  },
+  {
+    TraceId: Id_103,
+    SpanId: Id_105,
+    Name: http.request,
+    Resource: GET serverless-dummy-api:9005/function/HandlerNestedStructParam,
+    Service: Bootstrap-http-client,
+    Type: http,
+    ParentId: Id_104,
+    Tags: {
+      component: WebRequest,
+      http.method: GET,
+      http.status_code: 200,
+      http.url: http://serverless-dummy-api:9005/function/HandlerNestedStructParam,
+      runtime-id: Guid_27,
+      span.kind: client
+    },
+    Metrics: {
+      _dd.top_level: 1.0
+    }

@andrewlock

Copy link
Copy Markdown
Member Author

Benchmarks Report 🐌

Benchmarks for #3359 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 825μs 467ns 1.75μs 0.408 0 0 3.22 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 572μs 146ns 566ns 0 0 0 2.63 KB
#3359 WriteAndFlushEnrichedTraces net472 812μs 460ns 1.72μs 0.398 0 0 3.22 KB
#3359 WriteAndFlushEnrichedTraces netcoreapp3.1 564μs 351ns 1.36μs 0 0 0 2.63 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 196ns 0.202ns 0.727ns 0.0701 0 0 441 B
master AllCycleSimpleBody netcoreapp3.1 254ns 0.265ns 1.03ns 0.00587 0 0 440 B
master AllCycleMoreComplexBody net472 194ns 0.0749ns 0.27ns 0.0662 0 0 417 B
master AllCycleMoreComplexBody netcoreapp3.1 251ns 0.132ns 0.494ns 0.00568 0 0 416 B
master BodyExtractorSimpleBody net472 298ns 0.355ns 1.33ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 241ns 0.171ns 0.638ns 0.00374 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.7μs 17.2ns 64.3ns 1.21 0.0157 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13μs 11.5ns 44.4ns 0.09 0 0 6.75 KB
#3359 AllCycleSimpleBody net472 203ns 0.0535ns 0.193ns 0.0701 0 0 441 B
#3359 AllCycleSimpleBody netcoreapp3.1 252ns 0.196ns 0.733ns 0.00598 0 0 440 B
#3359 AllCycleMoreComplexBody net472 194ns 0.057ns 0.205ns 0.0662 0 0 417 B
#3359 AllCycleMoreComplexBody netcoreapp3.1 251ns 0.189ns 0.733ns 0.0058 0 0 416 B
#3359 BodyExtractorSimpleBody net472 289ns 0.161ns 0.601ns 0.0573 0 0 361 B
#3359 BodyExtractorSimpleBody netcoreapp3.1 241ns 0.191ns 0.714ns 0.00364 0 0 272 B
#3359 BodyExtractorMoreComplexBody net472 15.8μs 16.4ns 61.4ns 1.21 0.0157 0 7.62 KB
#3359 BodyExtractorMoreComplexBody netcoreapp3.1 12.9μs 6.2ns 23.2ns 0.0899 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown 🤷 Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 182μs 249ns 966ns 0.181 0 0 20.66 KB
#3359 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3359 SendRequest netcoreapp3.1 182μs 237ns 919ns 0.274 0 0 20.67 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.84μs 1.02ns 3.97ns 0.158 0.000923 0 995 B
master ExecuteNonQuery netcoreapp3.1 1.39μs 0.813ns 3.15ns 0.0133 0 0 984 B
#3359 ExecuteNonQuery net472 1.88μs 0.57ns 2.13ns 0.158 0.000945 0 995 B
#3359 ExecuteNonQuery netcoreapp3.1 1.51μs 0.633ns 2.37ns 0.013 0 0 984 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.43μs 0.758ns 2.84ns 0.191 0 0 1.2 KB
master CallElasticsearch netcoreapp3.1 1.53μs 0.529ns 2.05ns 0.0153 0 0 1.14 KB
master CallElasticsearchAsync net472 2.68μs 1.43ns 5.55ns 0.212 0 0 1.34 KB
master CallElasticsearchAsync netcoreapp3.1 1.64μs 0.533ns 2ns 0.0172 0 0 1.26 KB
#3359 CallElasticsearch net472 2.6μs 0.794ns 2.97ns 0.191 0 0 1.2 KB
#3359 CallElasticsearch netcoreapp3.1 1.54μs 1.1ns 4.25ns 0.0154 0 0 1.14 KB
#3359 CallElasticsearchAsync net472 2.68μs 0.998ns 3.73ns 0.213 0 0 1.34 KB
#3359 CallElasticsearchAsync netcoreapp3.1 1.59μs 0.41ns 1.48ns 0.0174 0 0 1.26 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.62μs 1.64ns 6.13ns 0.232 0 0 1.47 KB
master ExecuteAsync netcoreapp3.1 1.79μs 1.11ns 4.15ns 0.0187 0 0 1.39 KB
#3359 ExecuteAsync net472 2.71μs 1.9ns 7.1ns 0.233 0 0 1.47 KB
#3359 ExecuteAsync netcoreapp3.1 1.78μs 0.674ns 2.52ns 0.0186 0 0 1.39 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.91μs 2.51ns 9.73ns 0.445 0 0 2.82 KB
master SendAsync netcoreapp3.1 3.71μs 1.31ns 4.9ns 0.0352 0 0 2.65 KB
#3359 SendAsync net472 5.9μs 1.49ns 5.37ns 0.446 0 0 2.82 KB
#3359 SendAsync netcoreapp3.1 3.59μs 1.88ns 7.04ns 0.0358 0 0 2.65 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.2μs 3.37ns 12.6ns 0.297 0 0 1.88 KB
master EnrichedLog netcoreapp3.1 2.57μs 1.24ns 4.81ns 0.0256 0 0 1.91 KB
#3359 EnrichedLog net472 3.07μs 2.21ns 8.28ns 0.298 0 0 1.88 KB
#3359 EnrichedLog netcoreapp3.1 2.59μs 1.07ns 4.15ns 0.0258 0 0 1.91 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 152μs 147ns 570ns 0.677 0.226 0 4.72 KB
master EnrichedLog netcoreapp3.1 120μs 93.3ns 361ns 0.06 0 0 4.55 KB
#3359 EnrichedLog net472 151μs 193ns 747ns 0.677 0.226 0 4.72 KB
#3359 EnrichedLog netcoreapp3.1 120μs 274ns 1.06μs 0.0595 0 0 4.55 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.78μs 4.92ns 18.4ns 0.58 0.00287 0 3.65 KB
master EnrichedLog netcoreapp3.1 4.33μs 1.92ns 7.42ns 0.0545 0 0 3.98 KB
#3359 EnrichedLog net472 5.57μs 1.17ns 4.39ns 0.579 0.00278 0 3.65 KB
#3359 EnrichedLog netcoreapp3.1 4.36μs 1.53ns 5.74ns 0.0544 0 0 3.98 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.3μs 2.72ns 10.2ns 0.225 0 0 1.42 KB
master SendReceive netcoreapp3.1 1.77μs 0.612ns 2.37ns 0.0186 0 0 1.37 KB
#3359 SendReceive net472 2.34μs 1.31ns 5.06ns 0.225 0 0 1.42 KB
#3359 SendReceive netcoreapp3.1 1.82μs 0.678ns 2.54ns 0.0182 0 0 1.37 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.03μs 2.28ns 8.53ns 0.364 0 0 2.3 KB
master EnrichedLog netcoreapp3.1 4.15μs 1.31ns 4.72ns 0.0248 0 0 1.86 KB
#3359 EnrichedLog net472 5.03μs 2.09ns 8.11ns 0.364 0 0 2.3 KB
#3359 EnrichedLog netcoreapp3.1 4.37μs 1.26ns 4.7ns 0.0241 0 0 1.86 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 1.14μs 0.4ns 1.49ns 0.136 0 0 859 B
master StartFinishSpan netcoreapp3.1 935ns 0.34ns 1.27ns 0.0107 0 0 808 B
master StartFinishScope net472 1.35μs 0.996ns 3.73ns 0.149 0 0 939 B
master StartFinishScope netcoreapp3.1 1.07μs 0.46ns 1.78ns 0.0124 0 0 928 B
#3359 StartFinishSpan net472 1.16μs 0.443ns 1.66ns 0.136 0 0 859 B
#3359 StartFinishSpan netcoreapp3.1 904ns 0.34ns 1.27ns 0.0108 0 0 808 B
#3359 StartFinishScope net472 1.38μs 0.564ns 2.11ns 0.149 0 0 939 B
#3359 StartFinishScope netcoreapp3.1 1.06μs 0.453ns 1.69ns 0.0127 0 0 928 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.48μs 0.451ns 1.75ns 0.149 0 0 939 B
master RunOnMethodBegin netcoreapp3.1 1.12μs 0.576ns 2.23ns 0.0129 0 0 928 B
#3359 RunOnMethodBegin net472 1.49μs 0.414ns 1.55ns 0.148 0 0 939 B
#3359 RunOnMethodBegin netcoreapp3.1 1.22μs 0.33ns 1.23ns 0.0128 0 0 928 B

@andrewlock

Copy link
Copy Markdown
Member Author

Code Coverage Report 📊

✔️ Merging #3359 into master will not change line coverage
✔️ Merging #3359 into master will not change branch coverage
✔️ Merging #3359 into master will not change complexity

master #3359 Change
Lines 19602 / 27011 19574 / 27011
Lines % 73% 72% 0% ✔️
Branches 11633 / 16784 11613 / 16784
Branches % 69% 69% 0% ✔️
Complexity 18160 18160 0 ✔️

View the full report for further details:

Datadog.Trace Breakdown ✔️

master #3359 Change
Lines % 73% 72% 0% ✔️
Branches % 69% 69% 0% ✔️
Complexity 18160 18160 0 ✔️

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Iast.StackWalker 0% ✔️ 6% ✔️ 0 ✔️
Datadog.Trace.Debugger.PInvoke.DebuggerNativeMethods 6% ✔️ 0% ✔️ 0 ✔️
Datadog.Trace.ClrProfiler.AutoInstrumentation.Http.HttpClient.WinHttpHandler.WinHttpHandlerIntegration 100% ✔️ 0% ✔️ 0 ✔️

View the full reports for further details:

@bouwkast bouwkast left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

The diagram, images, and comments were all very helpful 👍

@andrewlock
andrewlock merged commit 79a69b7 into master Oct 19, 2022
@andrewlock
andrewlock deleted the andrew/serverless-tests branch October 19, 2022 08:38
@github-actions github-actions Bot added this to the vNext milestone Oct 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:serverless area:test-apps apps used to test integrations area:tests unit tests, integration tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants