Skip to content

[NO-TICKET] Print failure info when native extension setup fails#5657

Merged
ivoanjo merged 3 commits into
masterfrom
ivoanjo/improve-failure-debug
May 5, 2026
Merged

[NO-TICKET] Print failure info when native extension setup fails#5657
ivoanjo merged 3 commits into
masterfrom
ivoanjo/improve-failure-debug

Conversation

@ivoanjo

@ivoanjo ivoanjo commented May 1, 2026

Copy link
Copy Markdown
Member

What does this PR do?

When the native extension setup fails, the default mkmf error message is shown which unfortunately doesn't include any details to debug the failure.

This PR fixes that by wrapping the mkmf failure to print more info.

Motivation:

Investigating flaky build failures like
https://github.com/DataDog/dd-trace-rb/actions/runs/24427070812/job/71416807129?pr=5590 where we see the failure but lack the context to investigate.

Change log entry

Yes. Print failure info when native extension setup fails

Additional Notes:

N/A

How to test the change?

To test this locally, I was changing our extconf.rb files with something along the lines of

have_header("does_not_exist.h")
abort "forced failure"

Then running bundle exec rake clean compile to trigger.

@ivoanjo
[NO-TICKET] Print failure info when native extension setup fails
0600bc7 
**What does this PR do?**

When the native extension setup fails, the default mkmf error message
is shown which unfortunately doesn't include any details to debug the
failure.

This PR fixes that by wrapping the mkmf failure to print more info.

**Motivation:**

Investigating flaky build failures like
https://github.com/DataDog/dd-trace-rb/actions/runs/24427070812/job/71416807129?pr=5590
where we see the failure but lack the context to investigate.

**Change log entry**

Yes. Print failure info when native extension setup fails

**Additional Notes:**

N/A

**How to test the change?**

To test this locally, I was changing our `extconf.rb` files with
something along the lines of

```ruby
have_header("does_not_exist.h")
abort "forced failure"
```

Then running `bundle exec rake clean compile` to trigger.
@ivoanjo
ivoanjo requested review from a team as code owners May 1, 2026 09:35
@dd-octo-sts dd-octo-sts Bot added the profiling Involves Datadog profiling label May 1, 2026
@ivoanjo @claude
Fix spec on macOS
c393a71 
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented May 1, 2026 •
edited by datadog-prod-us1-4 Bot
Loading

Copy link
Copy Markdown
Contributor

Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
• Patch Coverage: 94.12%
• Overall Coverage: 97.21% (-0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ef25d67 | Docs | Datadog PR Page | Give us feedback!

@ivoanjo

ivoanjo commented May 1, 2026

Copy link
Copy Markdown
Member Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented May 1, 2026 •
edited
Loading

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-05-01 14:55:24 UTC ℹ️ Start processing command /merge

2026-05-01 14:55:35 UTC ℹ️ MergeQueue: waiting for PR to be ready

This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
It will be added to the queue as soon as checks pass and/or get approvals. View in MergeQueue UI.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2026-05-01 18:58:16 UTC ⚠️ MergeQueue: This merge request was unqueued

devflow unqueued this merge request: It did not become mergeable within the expected time

@pr-commenter

pr-commenter Bot commented May 5, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-05-05 09:17:07

Comparing candidate commit ef25d67 in PR branch ivoanjo/improve-failure-debug with baseline commit 524bafd in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

@ivoanjo
ivoanjo merged commit dc63047 into master May 5, 2026
591 of 593 checks passed
@ivoanjo
ivoanjo deleted the ivoanjo/improve-failure-debug branch May 5, 2026 09:40
@dd-octo-sts dd-octo-sts Bot added this to the 2.32.0 milestone May 5, 2026
@marcotc marcotc mentioned this pull request May 7, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Strech Strech Strech approved these changes
@p-datadog p-datadog p-datadog approved these changes

Assignees

No one assigned

Labels

mergequeue-status: removed profiling Involves Datadog profiling

Projects

None yet

Milestone

2.32.0

Development

Successfully merging this pull request may close these issues.

3 participants

@ivoanjo @Strech @p-datadog