fix(CI): resolve flaky tests by Leiyks · Pull Request #3560 · DataDog/dd-trace-php

Leiyks · 2026-01-08T13:39:17Z

This PR addresses several recurring flaky tests.

1. Kafka Integration Tests - Race Condition

Issue:
Kafka and Zookeeper start in parallel. Kafka attempts to connect before Zookeeper completes initialization.

Fix:

.gitlab/generate-common.php: Added Zookeeper readiness check to Kafka entrypoint (waits for ruok/imok response)
.gitlab/wait-for-service-ready.sh: Added Zookeeper service detection and native health check, increased Kafka settle time

2. CentOS Package Verification - Yum Timeouts

Issue:
verify centos failing with "Exiting on user cancel" - actually yum's 30-second stalled download timeout. Retries happened immediately, hitting same network issues.

Fix:

dockerfiles/verify_packages/centos/install.sh: Increased yum timeout to 300s, added 5-second delay between retries

3. Profiling Tests - PHP 7.X jit allocation

Issue:
allocation_jit_01.phpt failing on PHP 7.X versions even tho jit feature is available for 8.0+ versions.

Fix:

profiling/tests/phpt/allocation_jit_01.phpt: Added skip condition for PHP < 8.0 (consistent with other JIT tests)

4. Extension CI Tests - Valgrind Timeouts

Issue:
agent_sampling-standalone-asm_*.phpt timing out during valgrind run (second pass with memory checking). Valgrind causes slowdown which is enough to hit PHP 60s test timeout.

Fix:

tests/ext/background-sender/agent_sampling-standalone-asm_*.phpt: Increased maxIteration and loop number of iterations

codecov-commenter · 2026-01-08T13:49:45Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 61.82%. Comparing base (4a26042) to head (9c58f4d).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3560      +/-   ##
==========================================
- Coverage   61.90%   61.82%   -0.08%     
==========================================
  Files         140      140              
  Lines       13281    13281              
  Branches     1758     1758              
==========================================
- Hits         8221     8211      -10     
- Misses       4272     4279       +7     
- Partials      788      791       +3

see 3 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4a26042...9c58f4d. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

pr-commenter · 2026-01-08T15:04:26Z

Benchmarks [ profiler ]

Benchmark execution time: 2026-01-14 12:46:46

Comparing candidate commit 9c58f4d in PR branch leiyks/fix-flaky-ci-tests with baseline commit 4a26042 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 29 metrics, 7 unstable metrics.

… before starting kafka Signed-off-by: Alexandre Rulleau <[email protected]>

Signed-off-by: Alexandre Rulleau <[email protected]>

datadog-datadog-prod-us1 · 2026-01-13T10:43:14Z

⚠️ Tests

✨ Fix all issues with Cursor

⚠️ Warnings

❄️ 3 New flaky tests detected

testUserLoginFailureEvent from symfony-44-test.DDTrace\Tests\Integrations\Symfony\V4_4\AutomatedLoginEventsTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Symfony\V4_4\AutomatedLoginEventsTest::testUserLoginFailureEvent
Failed asserting that 0 matches expected 1.

tests/Integrations/Symfony/AutomatedLoginEventsTestSuite.php:74
tests/Common/RetryTraitVersionGeneric.php:28
phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106
testUserLoginSuccessEvent from symfony-44-test.DDTrace\Tests\Integrations\Symfony\V4_4\AutomatedLoginEventsTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Symfony\V4_4\AutomatedLoginEventsTest::testUserLoginSuccessEvent
Failed asserting that 0 matches expected 1.

tests/Integrations/Symfony/AutomatedLoginEventsTestSuite.php:56
tests/Common/RetryTraitVersionGeneric.php:28
phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106
testUserSignUp from symfony-44-test.DDTrace\Tests\Integrations\Symfony\V4_4\AutomatedLoginEventsTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Symfony\V4_4\AutomatedLoginEventsTest::testUserSignUp
Failed asserting that 0 matches expected 1.

tests/Integrations/Symfony/AutomatedLoginEventsTestSuite.php:94
tests/Common/RetryTraitVersionGeneric.php:28
phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106

🧪 2063 Tests failed

testSearchPhpBinaries from integration.DDTrace\Tests\Integration\PHPInstallerTest (Fix with Cursor)

testSimplePushAndProcess from laravel-58-test.DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest (Fix with Cursor)
testSimplePushAndProcess from laravel-8x-test.DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 69678f9d0000000071b550f9ceceeef1
tid: 69678f9d00000000
hexProcessTraceId: 71b550f9ceceeef1
hexProcessSpanId: cb182d32d115dd40
processTraceId: 8193544130917232369
processSpanId: 14634496685419912512
View all

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 9c58f4d | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

Signed-off-by: Alexandre Rulleau <[email protected]>

Leiyks changed the title ~~fix(test_integrations_kafka): properly wait for zookeeper to be ready…~~ fix(CI): resolve flaky tests Jan 8, 2026

Leiyks force-pushed the leiyks/fix-flaky-ci-tests branch 5 times, most recently from a8dc442 to 34a1dec Compare January 9, 2026 12:47

Leiyks marked this pull request as ready for review January 9, 2026 13:05

Leiyks requested review from a team as code owners January 9, 2026 13:05

Leiyks force-pushed the leiyks/fix-flaky-ci-tests branch from 34a1dec to 9648227 Compare January 9, 2026 13:21

Leiyks added 3 commits January 13, 2026 11:36

fix(test_integrations_kafka): properly wait for zookeeper to be ready…

ad87266

… before starting kafka Signed-off-by: Alexandre Rulleau <[email protected]>

fix(verify_centos): increase timeout to avoid failure

4f86ee4

Signed-off-by: Alexandre Rulleau <[email protected]>

fix(profiling_test): skip jit related test on 7.X

a95b1ff

Signed-off-by: Alexandre Rulleau <[email protected]>

Leiyks force-pushed the leiyks/fix-flaky-ci-tests branch from 9648227 to 88ce401 Compare January 13, 2026 10:36

realFlowControl mentioned this pull request Jan 13, 2026

fix(prof): use cached heap in alloc_prof_orig_* functions #3547

Merged

2 tasks

realFlowControl reviewed Jan 14, 2026

View reviewed changes

Comment thread .gitlab/generate-package.php Outdated

Leiyks force-pushed the leiyks/fix-flaky-ci-tests branch from 88ce401 to 803464e Compare January 14, 2026 12:31

fix(test_extension_ci): increase timeout for valgrind runs

9c58f4d

Signed-off-by: Alexandre Rulleau <[email protected]>

Leiyks force-pushed the leiyks/fix-flaky-ci-tests branch from 803464e to 9c58f4d Compare January 14, 2026 12:32

bwoebi approved these changes Jan 14, 2026

View reviewed changes

Leiyks merged commit 88a859c into master Jan 14, 2026
1995 of 2009 checks passed

Leiyks deleted the leiyks/fix-flaky-ci-tests branch January 14, 2026 13:05

github-actions Bot added this to the 1.16.0 milestone Jan 14, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(CI): resolve flaky tests#3560

fix(CI): resolve flaky tests#3560
Leiyks merged 4 commits intomasterfrom
leiyks/fix-flaky-ci-tests

Leiyks commented Jan 8, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Jan 8, 2026 •

edited

Loading

Uh oh!

pr-commenter Bot commented Jan 8, 2026 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1 Bot commented Jan 13, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Leiyks commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1. Kafka Integration Tests - Race Condition

2. CentOS Package Verification - Yum Timeouts

3. Profiling Tests - PHP 7.X jit allocation

4. Extension CI Tests - Valgrind Timeouts

Uh oh!

codecov-commenter commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

pr-commenter Bot commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks [ profiler ]

Uh oh!

datadog-datadog-prod-us1 Bot commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Leiyks commented Jan 8, 2026 •

edited

Loading

codecov-commenter commented Jan 8, 2026 •

edited

Loading

pr-commenter Bot commented Jan 8, 2026 •

edited

Loading

datadog-datadog-prod-us1 Bot commented Jan 13, 2026 •

edited

Loading