internal: Fixing `_dd.p.dm` decision maker collision on number 10. by yuanyuanzhao3 · Pull Request #2672 · DataDog/dd-trace-go

yuanyuanzhao3 · 2024-04-25T17:02:14Z

What does this PR do?

Fixing a collision in _dd.p.dm where number 10 was used both by data jobs and remote user rules.

An accompanying system test change PR is also made in DataDog/system-tests#2394.

** This is intended to be cherry-picked for patch release 1.63.1.**

Impact of the collision:
Small - feature won't be triggered unless allowed through a feature flag (currently disabled by default).

Motivation

Reviewer's Checklist

Changed code has unit tests for its functionality at or near 100% coverage.
System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
There is a benchmark for any new code, or changes to existing code.
If this interacts with the agent in a new way, a system test has been added.
Add an appropriate team label so this PR gets put in the right place for the release notes.
Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

darccio · 2024-04-26T11:15:42Z

Just for the record, parametric tests are failing due not being updated on main:

_ TestDynamicConfigSamplingRules.test_trace_sampling_rules_override_env[library_env0] _
[gw14] linux -- Python 3.9.19 /home/runner/work/dd-trace-go/dd-trace-go/venv/bin/python
self = <tests.parametric.test_dynamic_configuration.TestDynamicConfigSamplingRules object at 0x7f6970671a60>
library_env = {'DD_ENV': 'test_env', 'DD_INTERNAL_TELEMETRY_V2_ENABLED': '1', 'DD_REMOTE_CONFIG_POLL_INTERVAL_SECONDS': '0.2', 'DD_SERVICE': 'test_service', ...}
test_agent = <tests.parametric.conftest._TestAgentAPI object at 0x7f699d734100>
test_library = <utils.parametric._library_client.APMLibrary object at 0x7f699d6977f0>
    @parametrize(
        "library_env",
        [
            {
                **DEFAULT_ENVVARS,
                "DD_TRACE_SAMPLING_RULES": json.dumps([{"sample_rate": ENV_SAMPLING_RULE_RATE, "service": "*"}]),
            }
        ],
    )
    def test_trace_sampling_rules_override_env(self, library_env, test_agent, test_library):
        """The RC sampling rules should override the environment variable and decision maker is set appropriately.
    
        When RC is unset, the environment variable should be used.
        """
        RC_SAMPLING_RULE_RATE_CUSTOMER = 0.8
        RC_SAMPLING_RULE_RATE_DYNAMIC = 0.4
        assert RC_SAMPLING_RULE_RATE_CUSTOMER != ENV_SAMPLING_RULE_RATE
        assert RC_SAMPLING_RULE_RATE_DYNAMIC != ENV_SAMPLING_RULE_RATE
        assert RC_SAMPLING_RULE_RATE_CUSTOMER != DEFAULT_SAMPLE_RATE
        assert RC_SAMPLING_RULE_RATE_DYNAMIC != DEFAULT_SAMPLE_RATE
    
        trace = get_sampled_trace(test_library, test_agent, service="", name="env_name")
        assert_sampling_rate(trace, ENV_SAMPLING_RULE_RATE)
        # Make sure `_dd.p.dm` is set to "-3" (i.e., local RULE_RATE)
        span = trace[0]
        assert "_dd.p.dm" in span["meta"]
        # The "-" is a separating hyphen, not a minus sign.
        assert span["meta"]["_dd.p.dm"] == "-3"
    
        # Create a remote config entry with two rules at different sample rates.
        set_and_wait_rc(
            test_agent,
            config_overrides={
                "tracing_sampling_rules": [
                    {
                        "sample_rate": RC_SAMPLING_RULE_RATE_CUSTOMER,
                        "service": TEST_SERVICE,
                        "resource": "*",
                        "provenance": "customer",
                    },
                    {
                        "sample_rate": RC_SAMPLING_RULE_RATE_DYNAMIC,
                        "service": "*",
                        "resource": "*",
                        "provenance": "dynamic",
                    },
                ]
            },
        )
    
        trace = get_sampled_trace(test_library, test_agent, service=TEST_SERVICE, name="op_name")
        assert_sampling_rate(trace, RC_SAMPLING_RULE_RATE_CUSTOMER)
        # Make sure `_dd.p.dm` is set to "-10" (i.e., remote user rule)
        span = root_span(trace)
        assert "_dd.p.dm" in span["meta"]
>       assert span["meta"]["_dd.p.dm"] == "-10"
E       AssertionError: assert '-11' == '-10'
E         - -10
E         + -11
tests/parametric/test_dynamic_configuration.py:[62](https://github.com/DataDog/dd-trace-go/actions/runs/8836533580/job/24291020518#step:6:63)3: AssertionError

I think this is proof enough to assume the change is fine; it just needs to get updated in parametric-tests.

yuanyuanzhao3 · 2024-04-26T14:37:19Z

The system tests

Just for the record, parametric tests are failing due not being updated on main:

_ TestDynamicConfigSamplingRules.test_trace_sampling_rules_override_env[library_env0] _
[gw14] linux -- Python 3.9.19 /home/runner/work/dd-trace-go/dd-trace-go/venv/bin/python
self = <tests.parametric.test_dynamic_configuration.TestDynamicConfigSamplingRules object at 0x7f6970671a60>
library_env = {'DD_ENV': 'test_env', 'DD_INTERNAL_TELEMETRY_V2_ENABLED': '1', 'DD_REMOTE_CONFIG_POLL_INTERVAL_SECONDS': '0.2', 'DD_SERVICE': 'test_service', ...}
test_agent = <tests.parametric.conftest._TestAgentAPI object at 0x7f699d734100>
test_library = <utils.parametric._library_client.APMLibrary object at 0x7f699d6977f0>
    @parametrize(
        "library_env",
        [
            {
                **DEFAULT_ENVVARS,
                "DD_TRACE_SAMPLING_RULES": json.dumps([{"sample_rate": ENV_SAMPLING_RULE_RATE, "service": "*"}]),
            }
        ],
    )
    def test_trace_sampling_rules_override_env(self, library_env, test_agent, test_library):
        """The RC sampling rules should override the environment variable and decision maker is set appropriately.
    
        When RC is unset, the environment variable should be used.
        """
        RC_SAMPLING_RULE_RATE_CUSTOMER = 0.8
        RC_SAMPLING_RULE_RATE_DYNAMIC = 0.4
        assert RC_SAMPLING_RULE_RATE_CUSTOMER != ENV_SAMPLING_RULE_RATE
        assert RC_SAMPLING_RULE_RATE_DYNAMIC != ENV_SAMPLING_RULE_RATE
        assert RC_SAMPLING_RULE_RATE_CUSTOMER != DEFAULT_SAMPLE_RATE
        assert RC_SAMPLING_RULE_RATE_DYNAMIC != DEFAULT_SAMPLE_RATE
    
        trace = get_sampled_trace(test_library, test_agent, service="", name="env_name")
        assert_sampling_rate(trace, ENV_SAMPLING_RULE_RATE)
        # Make sure `_dd.p.dm` is set to "-3" (i.e., local RULE_RATE)
        span = trace[0]
        assert "_dd.p.dm" in span["meta"]
        # The "-" is a separating hyphen, not a minus sign.
        assert span["meta"]["_dd.p.dm"] == "-3"
    
        # Create a remote config entry with two rules at different sample rates.
        set_and_wait_rc(
            test_agent,
            config_overrides={
                "tracing_sampling_rules": [
                    {
                        "sample_rate": RC_SAMPLING_RULE_RATE_CUSTOMER,
                        "service": TEST_SERVICE,
                        "resource": "*",
                        "provenance": "customer",
                    },
                    {
                        "sample_rate": RC_SAMPLING_RULE_RATE_DYNAMIC,
                        "service": "*",
                        "resource": "*",
                        "provenance": "dynamic",
                    },
                ]
            },
        )
    
        trace = get_sampled_trace(test_library, test_agent, service=TEST_SERVICE, name="op_name")
        assert_sampling_rate(trace, RC_SAMPLING_RULE_RATE_CUSTOMER)
        # Make sure `_dd.p.dm` is set to "-10" (i.e., remote user rule)
        span = root_span(trace)
        assert "_dd.p.dm" in span["meta"]
>       assert span["meta"]["_dd.p.dm"] == "-10"
E       AssertionError: assert '-11' == '-10'
E         - -10
E         + -11
tests/parametric/test_dynamic_configuration.py:[62](https://github.com/DataDog/dd-trace-go/actions/runs/8836533580/job/24291020518#step:6:63)3: AssertionError

I think this is proof enough to assume the change is fine; it just needs to get updated in parametric-tests.

Thanks for approving the PR and cherrypicking it into 1.63.1. The system tests change is in DataDog/system-tests#2394.

One question, should the golang.yml test filter be v1.63.1, v1.64.0-dev or something else?

pr-commenter · 2024-04-26T18:24:33Z

Benchmarks

Benchmark execution time: 2024-04-26 18:24:22

Comparing candidate commit 2ba3f40 in PR branch yuanyuan.zhao/remote-rules-dm-collision-fix with baseline commit 4e7fb35 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 43 metrics, 1 unstable metrics.

Fixing _dd.p.dm decision maker collision on number 10.

dc39e41

yuanyuanzhao3 added the team:apm-go label Apr 25, 2024

yuanyuanzhao3 requested a review from a team as a code owner April 25, 2024 17:02

yuanyuanzhao3 requested a review from dianashevchenko April 25, 2024 17:09

dianashevchenko approved these changes Apr 26, 2024

View reviewed changes

darccio mentioned this pull request Apr 26, 2024

[v1.63.1] internal: Fixing _dd.p.dm decision maker collision on number 10 #2673

Merged

6 tasks

eliottness approved these changes Apr 26, 2024

View reviewed changes

Merge branch 'main' into yuanyuan.zhao/remote-rules-dm-collision-fix

2ba3f40

darccio merged commit 155ef2d into main Apr 29, 2024

darccio deleted the yuanyuan.zhao/remote-rules-dm-collision-fix branch April 29, 2024 10:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

internal: Fixing `_dd.p.dm` decision maker collision on number 10.#2672

internal: Fixing `_dd.p.dm` decision maker collision on number 10.#2672
darccio merged 2 commits into
mainfrom
yuanyuan.zhao/remote-rules-dm-collision-fix

yuanyuanzhao3 commented Apr 25, 2024

Uh oh!

darccio commented Apr 26, 2024

Uh oh!

yuanyuanzhao3 commented Apr 26, 2024

Uh oh!

pr-commenter Bot commented Apr 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

yuanyuanzhao3 commented Apr 25, 2024

What does this PR do?

Motivation

Reviewer's Checklist

Uh oh!

darccio commented Apr 26, 2024

Uh oh!

yuanyuanzhao3 commented Apr 26, 2024

Uh oh!

pr-commenter Bot commented Apr 26, 2024

Benchmarks

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants