Skip to content

RUM-6596 Fix the execution of legacy instrumentation tests in CI#2329

Merged
mariusc83 merged 1 commit into
developfrom
mconstantin/rum-6596/fix-legacy-integration-tests-ci-runtime
Oct 21, 2024
Merged

RUM-6596 Fix the execution of legacy instrumentation tests in CI#2329
mariusc83 merged 1 commit into
developfrom
mconstantin/rum-6596/fix-legacy-integration-tests-ci-runtime

Conversation

@mariusc83

@mariusc83 mariusc83 commented Oct 17, 2024

Copy link
Copy Markdown
Member

What does this PR do?

Problem Overview:

  • Legacy Integration Tests:

    • These tests are executed on MacOS Sonoma runners through GitLab CI.
    • They are experiencing silent failures, meaning the CI job completes as "green" or successful, even when tests fail.
  • Silent Failure Cause:

    • The silent failure issue primarily arises from the method of test execution.
    • Post-test execution necessitates killing the emulator to ensure the process exits properly.
  • Script Execution Approach:

    • The current approach involves executing shell scripts with deliberate error suppression (set +e).
    • This suppression is necessary to guarantee the emulator is terminated, even in the presence of errors.
  • GitLab CI Behavior:

    • GitLab executes shell scripts sequentially, processing one line at a time.
    • Exit codes are not propagated between lines, complicating error handling.
    • To manage exit codes, they must be persisted in a variable within the same line of execution.
  • Tools and Execution Challenges:

    • This method works effectively when executing tests via gradle.
    • However, it presents issues when executing tests directly through adb shell.

Motivation

What inspired you to submit this pull request?

Additional Notes

Anything else we should know when reviewing?

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Make sure you discussed the feature or bugfix with the maintaining team in an Issue
  • Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

@mariusc83
mariusc83 force-pushed the mconstantin/rum-6596/fix-legacy-integration-tests-ci-runtime branch 2 times, most recently from 2cbac15 to 5c47222 Compare October 18, 2024 09:26
@mariusc83 mariusc83 self-assigned this Oct 18, 2024
@mariusc83 mariusc83 changed the title RUM-6596 Make sure the legacy integration test fail the CI job if fai… RUM-6596 Fix the execution of legacy instrumentation tests in CI Oct 18, 2024
@codecov-commenter

codecov-commenter commented Oct 18, 2024

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.41%. Comparing base (d7e640c) to head (953b791).
Report is 2 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2329      +/-   ##
===========================================
- Coverage    70.43%   70.41%   -0.03%     
===========================================
  Files          739      739              
  Lines        27538    27538              
  Branches      4616     4616              
===========================================
- Hits         19396    19389       -7     
- Misses        6859     6865       +6     
- Partials      1283     1284       +1     

see 26 files with indirect coverage changes

@mariusc83
mariusc83 force-pushed the mconstantin/rum-6596/fix-legacy-integration-tests-ci-runtime branch 5 times, most recently from 25df38d to 429848e Compare October 21, 2024 09:52
@mariusc83
mariusc83 marked this pull request as ready for review October 21, 2024 10:48
@mariusc83
mariusc83 requested review from a team as code owners October 21, 2024 10:48
@mariusc83
mariusc83 force-pushed the mconstantin/rum-6596/fix-legacy-integration-tests-ci-runtime branch from 429848e to 953b791 Compare October 21, 2024 13:07
@mariusc83
mariusc83 merged commit c9439b5 into develop Oct 21, 2024
@mariusc83
mariusc83 deleted the mconstantin/rum-6596/fix-legacy-integration-tests-ci-runtime branch October 21, 2024 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants