Skip to content

Fix #expect_in_fork helper to check mock expectations#5790

Merged
eregon merged 1 commit into
masterfrom
fix-expect_in_fork-to-verify-mocks
May 21, 2026
Merged

Fix #expect_in_fork helper to check mock expectations#5790
eregon merged 1 commit into
masterfrom
fix-expect_in_fork-to-verify-mocks

Conversation

@eregon

@eregon eregon commented May 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Member
  • Before this, all mock expectations would be ignored,
    since they are checked after the after hooks run,
    but the fork only ran the passed block and stopped before any after hook.
  • RSpec::Mocks.with_temporary_scope creates a separate scope
    so only child process mock expectations are checked there.
    RSpec::Mocks.with_temporary_scope does the same as RSpec::Mocks.verify after calling the block.

What does this PR do?
What the title says.

Motivation:

Change log entry

None

Additional Notes:

How to test the change?

I added a spec for SynchronizationHelpers#expect_in_fork and verified it fails without the fix.
I also did some manual tests.

@eregon
eregon requested a review from a team as a code owner May 20, 2026 10:33
@dd-octo-sts dd-octo-sts Bot added the dev/testing Involves testing processes (e.g. RSpec) label May 20, 2026
chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed May 20, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d673afa176

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread spec/support/synchronization_helpers.rb Outdated
@eregon
eregon force-pushed the fix-expect_in_fork-to-verify-mocks branch 4 times, most recently from ff81a58 to ceaa0ce Compare May 20, 2026 10:52
@eregon
eregon enabled auto-merge May 20, 2026 10:52
TonyCTHsu
TonyCTHsu approved these changes May 20, 2026
View reviewed changes

@TonyCTHsu TonyCTHsu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome!

@eregon

eregon commented May 20, 2026

Copy link
Copy Markdown
Member Author

Thanks to @ivoanjo for helping me debug this one, I was starting to doubt whether anything works in RSpec but it turns out it was a bug of that helper.

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented May 20, 2026 •
edited by datadog-prod-us1-4 Bot
Loading

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 68.42%
• Overall Coverage: 97.08% (-0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 2c3cd94 | Docs | Datadog PR Page | Give us feedback!

@eregon eregon mentioned this pull request May 20, 2026
Merged
@eregon

eregon commented May 20, 2026

Copy link
Copy Markdown
Member Author

Pre-existing transient probably?
https://github.com/DataDog/dd-trace-rb/actions/runs/26181294848/job/77024930069?pr=5790

@ivoanjo

ivoanjo commented May 21, 2026

Copy link
Copy Markdown
Member

Pre-existing transient probably?

Yeah looks suspiciously related to #5795 -- maybe rebase on top of master?

@eregon
Fix #expect_in_fork helper to check mock expectations
2c3cd94 
* Before this, all mock expectations would be ignored,
  since they are checked after the `after` hooks run,
  but the fork only ran the passed block and stopped before any `after` hook.
* RSpec::Mocks.with_temporary_scope creates a separate scope
  so only child process mock expectations are checked there.
  RSpec::Mocks.with_temporary_scope does the same RSpec::Mocks.verify after calling the block.
@eregon
eregon force-pushed the fix-expect_in_fork-to-verify-mocks branch from 01d8a09 to 2c3cd94 Compare May 21, 2026 09:26
@pr-commenter

pr-commenter Bot commented May 21, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-05-21 09:52:13

Comparing candidate commit 2c3cd94 in PR branch fix-expect_in_fork-to-verify-mocks with baseline commit 50fc341 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

@eregon
eregon merged commit b5a389d into master May 21, 2026
308 checks passed
@eregon
eregon deleted the fix-expect_in_fork-to-verify-mocks branch May 21, 2026 09:53
@dd-octo-sts dd-octo-sts Bot added this to the 2.34.0 milestone May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments
@TonyCTHsu TonyCTHsu TonyCTHsu approved these changes
@p-datadog p-datadog p-datadog approved these changes

Assignees

No one assigned

Labels

dev/testing Involves testing processes (e.g. RSpec)

Projects

None yet

Milestone

2.34.0

Development

Successfully merging this pull request may close these issues.

4 participants

@eregon @ivoanjo @TonyCTHsu @p-datadog