Skip to content

[Test] Updates for opentelemetry test, use golang_cross in GH build cache key#6282

Merged
titpetric merged 2 commits into
masterfrom
test/opentelemetry-improvements
May 16, 2024
Merged

[Test] Updates for opentelemetry test, use golang_cross in GH build cache key#6282
titpetric merged 2 commits into
masterfrom
test/opentelemetry-improvements

Conversation

@titpetric

@titpetric titpetric commented May 15, 2024

Copy link
Copy Markdown
Contributor

User description

Github Actions:

  • GH go build cache key now has golang_cross to prevent conflicts between 1.21/1.22 caches

Opentelemetry:

  • Opentelemetry now all in /ci/tests/opentelemetry (Taskfile, tracetest moved out of .taskfiles)
  • Updates to opentelemetry collector from 0.80.0 to 0.100.0
  • Updates tracetest to 1.0.0
  • Cleans up configs and docker compose environment

If you have tracetest installed, delete it before running task opentelemetry.


PR Type

enhancement, tests


Description

  • Updated GitHub Actions to use a more specific cache key by including golang_cross version to prevent conflicts.
  • Removed old opentelemetry task configuration and introduced a new structured Taskfile in a different directory for enhanced management.
  • Updated Docker build commands in Taskfile to use the latest Go version and corresponding golang-cross image.
  • Introduced a new Taskfile for opentelemetry with comprehensive commands for setting up and running e2e tests.
  • Updated the OpenTelemetry collector configuration to include new settings and simplified the configuration structure.
  • Enhanced Tyk environment configurations by adding debug log level and specifying the storage host.
  • Overhauled Docker Compose configurations, updating service images, and removing unused network configurations.
  • Updated the Tracetest Taskfile to support new versioning and installation procedures.

Changes walkthrough 📝

Relevant files
Configuration changes
release.yml
Update GitHub Actions Cache Key Configuration                       

.github/workflows/release.yml

  • Updated the cache key and restore-keys in GitHub Actions to include
    golang_cross version, enhancing specificity and preventing conflicts
    between different Go versions.
  • +2/-2     
    opentelemetry.yml
    Remove Opentelemetry Task Configuration                                   

    .taskfiles/opentelemetry.yml

    • Removed the entire opentelemetry task configuration.
    +0/-49   
    Taskfile.yml
    Update Taskfile with New Opentelemetry Path and Go Version

    Taskfile.yml

  • Updated the inclusion path for opentelemetry tasks to a new directory.
  • Updated Docker build commands to use Go version 1.22 and corresponding
    golang-cross image.
  • +5/-3     
    collector.config.yml
    Update OpenTelemetry Collector Configuration                         

    ci/tests/tracing/configs/otelcollector/collector.config.yml

  • Updated the OpenTelemetry collector configuration, including new
    logging exporter settings and simplified structure.
  • +22/-20 
    tyk.env
    Enhance Tyk Environment Configuration                                       

    ci/tests/tracing/configs/tyk.env

  • Added debug log level and storage host configuration to Tyk
    environment settings.
  • +3/-1     
    docker-compose.yml
    Update Docker Compose Configuration for Tracing Tests       

    ci/tests/tracing/docker-compose.yml

  • Updated various service configurations, including new image versions
    and simplified service dependencies.
  • Removed unused network configurations.
  • +31/-38 
    Enhancement
    Taskfile.yml
    Introduce New Opentelemetry Taskfile for E2E Tests             

    ci/tests/tracing/Taskfile.yml

  • Added a new Taskfile for opentelemetry with updated commands and
    structure for running e2e tests.
  • +55/-0   
    Taskfile.yml
    Update Tracetest Taskfile with New Installation Tasks       

    ci/tests/tracing/tracetest/Taskfile.yml

  • Updated Tracetest Taskfile to include version variable and new
    installation tasks.
  • +12/-5   

    💡 PR-Agent usage:
    Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    @titpetric
    titpetric requested review from a team as code owners May 15, 2024 16:34
    @github-actions

    Copy link
    Copy Markdown
    Contributor

    API Changes

    no api changes detected

    @github-actions

    Copy link
    Copy Markdown
    Contributor

    PR Description updated to latest commit (fb8b819)

    @github-actions

    Copy link
    Copy Markdown
    Contributor

    PR Review 🔍

    ⏱️ Estimated effort to review [1-5]

    3, because the PR involves multiple files and configurations across different technologies (GitHub Actions, Docker, Taskfile, and OpenTelemetry). Understanding the implications of these changes requires a good grasp of CI/CD pipelines, containerization, and telemetry.

    🧪 Relevant tests

    No

    ⚡ Possible issues

    Possible Bug: The new cache key in the GitHub Actions workflow includes ${{ matrix.golang_cross }}, but there is no indication that matrix.golang_cross is defined in the workflow matrix. This could lead to incorrect or unresolved cache key values.

    🔒 Security concerns

    No

    Code feedback:
    relevant file.github/workflows/release.yml
    suggestion      

    Ensure that matrix.golang_cross is defined in your workflow matrix to avoid issues with the cache key resolution. If it's not defined, you might need to add it or adjust the cache key to use an existing variable. [important]

    relevant linekey: ${{ runner.os }}-${{ matrix.golang_cross }}-go-${{ hashFiles('**/go.sum') }}

    @github-actions

    Copy link
    Copy Markdown
    Contributor

    PR Code Suggestions ✨

    CategorySuggestion                                                                                                                                    Score
    Best practice
    Specify explicit protocol settings for OTLP to avoid unsuitable defaults

    Specify explicit values for the grpc and http protocols under the otlp receiver to avoid
    potential misconfigurations or defaults that may not be suitable for production
    environments.

    ci/tests/tracing/configs/otelcollector/collector.config.yml [3-6]

     otlp:
       protocols:
         grpc:
    +      endpoint: 0.0.0.0:4317
         http:
    +      endpoint: 0.0.0.0:4318
     
    Suggestion importance[1-10]: 8

    Why: This suggestion is highly relevant as it addresses potential misconfigurations in telemetry settings, which is crucial for ensuring correct and secure data transmission in production environments. The suggestion to specify explicit values enhances clarity and security.

    8
    Enhancement
    Add a fallback restore key to improve cache hits in GitHub Actions

    Consider adding a fallback restore key for the cache in the GitHub Actions workflow to
    improve cache hits when the hash of go.sum changes but the Golang version and OS remain
    the same. This can reduce build times by increasing the likelihood of a cache hit.

    .github/workflows/release.yml [75]

     restore-keys: |
       ${{ runner.os }}-${{ matrix.golang_cross }}-${{ hashFiles('**/go.sum') }}
    +  ${{ runner.os }}-${{ matrix.golang_cross }}-
     
    Suggestion importance[1-10]: 7

    Why: The suggestion correctly identifies an enhancement for caching strategy in GitHub Actions, which could improve build efficiency by providing a more robust cache fallback mechanism.

    7

    @titpetric
    titpetric enabled auto-merge (squash) May 15, 2024 16:48
    @titpetric
    titpetric force-pushed the test/opentelemetry-improvements branch from fb8b819 to 6944f19 Compare May 16, 2024 14:29
    @titpetric
    titpetric disabled auto-merge May 16, 2024 14:38
    @titpetric
    titpetric force-pushed the test/opentelemetry-improvements branch from e74cd32 to 370b210 Compare May 16, 2024 14:42
    @sonarqubecloud

    Copy link
    Copy Markdown

    Please retry analysis of this Pull-Request directly on SonarCloud

    @github-actions

    Copy link
    Copy Markdown
    Contributor

    💥 CI tests failed 🙈

    git-state

    all ok

    Please look at the run or in the Checks tab.

    @sonarqubecloud

    Copy link
    Copy Markdown

    Quality Gate Passed Quality Gate passed

    Issues
    0 New issues
    0 Accepted issues

    Measures
    0 Security Hotspots
    No data about Coverage
    0.0% Duplication on New Code

    See analysis details on SonarCloud

    @titpetric
    titpetric force-pushed the test/opentelemetry-improvements branch from 370b210 to e414a8b Compare May 16, 2024 16:33
    @titpetric
    titpetric enabled auto-merge (squash) May 16, 2024 16:33
    @sonarqubecloud

    Copy link
    Copy Markdown

    Quality Gate Passed Quality Gate passed

    Issues
    0 New issues
    0 Accepted issues

    Measures
    0 Security Hotspots
    No data about Coverage
    0.0% Duplication on New Code

    See analysis details on SonarCloud

    @titpetric
    titpetric merged commit e08594d into master May 16, 2024
    @titpetric
    titpetric deleted the test/opentelemetry-improvements branch May 16, 2024 16:54
    nerdydread pushed a commit that referenced this pull request Sep 6, 2024
    …ache key (#6282)
    
    ### **User description**
    Github Actions:
    
    - GH go build cache key now has golang_cross to prevent conflicts
    between 1.21/1.22 caches
    
    Opentelemetry:
    
    - Opentelemetry now all in /ci/tests/opentelemetry (Taskfile, tracetest
    moved out of .taskfiles)
    - Updates to opentelemetry collector from 0.80.0 to 0.100.0
    - Updates tracetest to 1.0.0
    - Cleans up configs and docker compose environment
    
    If you have tracetest installed, delete it before running `task
    opentelemetry`.
    
    
    ___
    
    ### **PR Type**
    enhancement, tests
    
    
    ___
    
    ### **Description**
    - Updated GitHub Actions to use a more specific cache key by including
    `golang_cross` version to prevent conflicts.
    - Removed old opentelemetry task configuration and introduced a new
    structured Taskfile in a different directory for enhanced management.
    - Updated Docker build commands in Taskfile to use the latest Go version
    and corresponding golang-cross image.
    - Introduced a new Taskfile for opentelemetry with comprehensive
    commands for setting up and running e2e tests.
    - Updated the OpenTelemetry collector configuration to include new
    settings and simplified the configuration structure.
    - Enhanced Tyk environment configurations by adding debug log level and
    specifying the storage host.
    - Overhauled Docker Compose configurations, updating service images, and
    removing unused network configurations.
    - Updated the Tracetest Taskfile to support new versioning and
    installation procedures.
    
    
    ___
    
    
    
    ### **Changes walkthrough** 📝
    <table><thead><tr><th></th><th align="left">Relevant
    files</th></tr></thead><tbody><tr><td><strong>Configuration changes
    </strong></td><td><table>
    <tr>
      <td>
        <details>
    <summary><strong>release.yml</strong><dd><code>Update GitHub Actions
    Cache Key Configuration</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
    &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
    <hr>
    
    .github/workflows/release.yml
    <li>Updated the cache key and restore-keys in GitHub Actions to include
    <br><code>golang_cross</code> version, enhancing specificity and
    preventing conflicts <br>between different Go versions.<br>
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6282/files#diff-87db21a973eed4fef5f32b267aa60fcee5cbdf03c67fafdc2a9b553bb0b15f34">+2/-2</a>&nbsp;
    &nbsp; &nbsp; </td>
    </tr>                    
    
    <tr>
      <td>
        <details>
    <summary><strong>opentelemetry.yml</strong><dd><code>Remove
    Opentelemetry Task Configuration</code>&nbsp; &nbsp; &nbsp; &nbsp;
    &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
    &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
    <hr>
    
    .taskfiles/opentelemetry.yml
    - Removed the entire opentelemetry task configuration.
    
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6282/files#diff-3a4b761dd20bd83804466e1295ad58db13b87b5e35e6120efc59c2b7bfaa27e1">+0/-49</a>&nbsp;
    &nbsp; </td>
    </tr>                    
    
    <tr>
      <td>
        <details>
    <summary><strong>Taskfile.yml</strong><dd><code>Update Taskfile with New
    Opentelemetry Path and Go Version</code></dd></summary>
    <hr>
    
    Taskfile.yml
    <li>Updated the inclusion path for opentelemetry tasks to a new
    directory.<br> <li> Updated Docker build commands to use Go version 1.22
    and corresponding <br>golang-cross image.<br>
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6282/files#diff-cd2d359855d0301ce190f1ec3b4c572ea690c83747f6df61c9340720e3d2425e">+5/-3</a>&nbsp;
    &nbsp; &nbsp; </td>
    </tr>                    
    
    <tr>
      <td>
        <details>
    <summary><strong>collector.config.yml</strong><dd><code>Update
    OpenTelemetry Collector Configuration</code>&nbsp; &nbsp; &nbsp; &nbsp;
    &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
    </dd></summary>
    <hr>
    
    ci/tests/tracing/configs/otelcollector/collector.config.yml
    <li>Updated the OpenTelemetry collector configuration, including new
    <br>logging exporter settings and simplified structure.<br>
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6282/files#diff-7ed5aea20a5cf1666f7de8ed60b105467e4a6be7a137d5c145e58f772803b690">+22/-20</a>&nbsp;
    </td>
    </tr>                    
    
    <tr>
      <td>
        <details>
    <summary><strong>tyk.env</strong><dd><code>Enhance Tyk Environment
    Configuration</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
    &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
    &nbsp; &nbsp; &nbsp; </dd></summary>
    <hr>
    
    ci/tests/tracing/configs/tyk.env
    <li>Added debug log level and storage host configuration to Tyk
    <br>environment settings.<br>
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6282/files#diff-69098dd1c6ed48bb914d7d4534673f9f42d146b7ecf37fda7c494886215bea1c">+3/-1</a>&nbsp;
    &nbsp; &nbsp; </td>
    </tr>                    
    
    <tr>
      <td>
        <details>
    <summary><strong>docker-compose.yml</strong><dd><code>Update Docker
    Compose Configuration for Tracing Tests</code>&nbsp; &nbsp; &nbsp;
    &nbsp; </dd></summary>
    <hr>
    
    ci/tests/tracing/docker-compose.yml
    <li>Updated various service configurations, including new image versions
    <br>and simplified service dependencies.<br> <li> Removed unused network
    configurations.<br>
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6282/files#diff-ca0b13836790f0db90a490b5000a738787f3d5d839313d456ec9f94b33dac7ee">+31/-38</a>&nbsp;
    </td>
    </tr>                    
    </table></td></tr><tr><td><strong>Enhancement
    </strong></td><td><table>
    <tr>
      <td>
        <details>
    <summary><strong>Taskfile.yml</strong><dd><code>Introduce New
    Opentelemetry Taskfile for E2E Tests</code>&nbsp; &nbsp; &nbsp; &nbsp;
    &nbsp; &nbsp; &nbsp; </dd></summary>
    <hr>
    
    ci/tests/tracing/Taskfile.yml
    <li>Added a new Taskfile for opentelemetry with updated commands and
    <br>structure for running e2e tests.<br>
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6282/files#diff-d7d2dd36087d885290fa602e3e810f648e36c0ebf71becf09fcd68e73371ac96">+55/-0</a>&nbsp;
    &nbsp; </td>
    </tr>                    
    
    <tr>
      <td>
        <details>
    <summary><strong>Taskfile.yml</strong><dd><code>Update Tracetest
    Taskfile with New Installation Tasks</code>&nbsp; &nbsp; &nbsp; &nbsp;
    </dd></summary>
    <hr>
    
    ci/tests/tracing/tracetest/Taskfile.yml
    <li>Updated Tracetest Taskfile to include version variable and new
    <br>installation tasks.<br>
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6282/files#diff-1ea785f38e54f75865fed6e19e96dff08ba102c80a3e438ae2e7d54b8e435af2">+12/-5</a>&nbsp;
    &nbsp; </td>
    </tr>                    
    </table></td></tr></tr></tbody></table>
    
    ___
    
    > 💡 **PR-Agent usage**:
    >Comment `/help` on the PR to get a list of all available PR-Agent tools
    and their descriptions
    
    ---------
    
    Co-authored-by: Tit Petric <[email protected]>
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Projects

    None yet

    Development

    Successfully merging this pull request may close these issues.

    3 participants