Skip to content

Commit 657b4f9

Browse files
BridgeARBridgeAR
authored
ci: fix coverage reports and prevent detecting fixture files (#7391)
Our test optimization product is using fixture files that are automatically picked up by CodeCov. Instead of having to declare our coverage files in all CI scripts, this just renames these fixture files, since they are just imported to read their content. To prevent future issues, a small lint job is added that prevents names that might be detected as coverage. The scripts are now way simpler due to the fix and a dynamic config is used for nyc. While this officially 'decreases' our coverage, the opposite is actually the case. We now track 9000 more lines and 200 files of which we cover 6000 lines. That is why our overall coverage drops while we actually properly handle the coverage now. All our CodeCov uploaded files are now correct and none create errors. This PR makes our coverage way more solid and is a first step towards also tracking the coverage of integration tests. Coverage is now tracked in individual folders per Node.js version and plugin name. That way CodeCov is merging the coverage instead of nyc and we never replace existing coverage.
1 parent d19c433 commit 657b4f9

File tree

29 files changed

+319
-111
lines changed

29 files changed

+319
-111
lines changed

‎.github/actions/coverage/action.yml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
name: coverage
2+
description: Verify coverage output and upload it to Codecov.
3+
4+
inputs:
5+
flags:
6+
description: "Codecov flags value"
7+
required: false
8+
report-dir:
9+
description: "Coverage report directory (defaults to 'coverage')"
10+
required: false
11+
default: coverage
12+
13+
runs:
14+
using: composite
15+
steps:
16+
- name: Verify coverage output
17+
shell: bash
18+
run: node scripts/verify-coverage.js --flags "${{ inputs.flags }}"
19+
20+
- name: Upload coverage to Codecov
21+
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
22+
with:
23+
flags: ${{ inputs.flags }}

‎.github/actions/instrumentations/test/action.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,9 @@ runs:
1414
- uses: ./.github/actions/node/latest
1515
- run: yarn test:instrumentations:ci
1616
shell: bash
17-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
17+
- uses: ./.github/actions/coverage
18+
with:
19+
flags: instrumentations-${{ github.job }}
1820
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
1921
if: always() && github.actor != 'dependabot[bot]'
2022
with:

‎.github/actions/plugins/test-and-upstream/action.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,9 @@ runs:
1919
shell: bash
2020
- run: yarn test:plugins:upstream
2121
shell: bash
22-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
22+
- uses: ./.github/actions/coverage
23+
with:
24+
flags: plugins-test-and-upstream-${{ github.job }}
2325
- if: always()
2426
uses: ./.github/actions/testagent/logs
2527
with:

‎.github/actions/plugins/test/action.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,9 @@ runs:
1515
- uses: ./.github/actions/node/latest
1616
- run: yarn test:plugins:ci
1717
shell: bash
18-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
18+
- uses: ./.github/actions/coverage
19+
with:
20+
flags: plugins-${{ github.job }}
1921
- if: always()
2022
uses: ./.github/actions/testagent/logs
2123
with:

‎.github/actions/plugins/upstream/action.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,9 @@ runs:
1515
- uses: ./.github/actions/node/latest
1616
- run: yarn test:plugins:upstream
1717
shell: bash
18-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
18+
- uses: ./.github/actions/coverage
19+
with:
20+
flags: plugins-upstream-${{ github.job }}
1921
- if: always()
2022
uses: ./.github/actions/testagent/logs
2123
with:

‎.github/workflows/aiguard.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ jobs:
2424
- uses: ./.github/actions/node/latest
2525
- uses: ./.github/actions/install
2626
- run: yarn test:aiguard:ci
27-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
27+
- uses: ./.github/actions/coverage
2828
with:
2929
flags: aiguard-macos
3030
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -47,7 +47,7 @@ jobs:
4747
- run: yarn test:aiguard:ci
4848
- uses: ./.github/actions/node/latest
4949
- run: yarn test:aiguard:ci
50-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
50+
- uses: ./.github/actions/coverage
5151
with:
5252
flags: aiguard-ubuntu
5353

@@ -61,7 +61,7 @@ jobs:
6161
with:
6262
cache: 'true'
6363
- run: yarn test:aiguard:ci
64-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
64+
- uses: ./.github/actions/coverage
6565
with:
6666
flags: aiguard-windows
6767
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1

‎.github/workflows/apm-capabilities.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ jobs:
3030
- uses: ./.github/actions/node/latest
3131
- uses: ./.github/actions/install
3232
- run: yarn test:trace:core:ci
33-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
33+
- uses: ./.github/actions/coverage
3434
with:
3535
flags: apm-capabilities-tracing-macos
3636
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -52,7 +52,7 @@ jobs:
5252
- run: yarn test:trace:core:ci
5353
- uses: ./.github/actions/node/latest
5454
- run: yarn test:trace:core:ci
55-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
55+
- uses: ./.github/actions/coverage
5656
with:
5757
flags: apm-capabilities-tracing-ubuntu
5858
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -70,7 +70,7 @@ jobs:
7070
with:
7171
cache: 'true'
7272
- run: yarn test:trace:core:ci
73-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
73+
- uses: ./.github/actions/coverage
7474
with:
7575
flags: apm-capabilities-tracing-windows
7676
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1

‎.github/workflows/apm-integrations.yml‎

Lines changed: 17 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ jobs:
7878
uses: ./.github/actions/testagent/logs
7979
with:
8080
suffix: plugins-${{ github.job }}-${{ matrix.node-version }}-${{ matrix.range_clean }}
81-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
81+
- uses: ./.github/actions/coverage
8282
with:
8383
flags: apm-integrations-aerospike-${{ matrix.node-version }}-${{ matrix.range_clean }}
8484
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -211,7 +211,7 @@ jobs:
211211
- run: yarn test:plugins:ci
212212
- uses: ./.github/actions/node/latest
213213
- run: yarn test:plugins:ci
214-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
214+
- uses: ./.github/actions/coverage
215215
with:
216216
flags: apm-integrations-child-process
217217
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -280,7 +280,7 @@ jobs:
280280
uses: ./.github/actions/testagent/logs
281281
with:
282282
suffix: plugins-${{ github.job }}-${{ matrix.node-version }}
283-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
283+
- uses: ./.github/actions/coverage
284284
with:
285285
flags: apm-integrations-confluentinc-kafka-javascript-${{ matrix.node-version }}
286286
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -342,7 +342,7 @@ jobs:
342342
- uses: ./.github/actions/install
343343
- run: yarn config set ignore-engines true
344344
- run: yarn test:plugins:ci --ignore-engines
345-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
345+
- uses: ./.github/actions/coverage
346346
with:
347347
flags: apm-integrations-couchbase-${{ matrix.node-version }}
348348
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -381,7 +381,7 @@ jobs:
381381
uses: ./.github/actions/testagent/logs
382382
with:
383383
suffix: plugins-${{ github.job }}
384-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
384+
- uses: ./.github/actions/coverage
385385
with:
386386
flags: apm-integrations-dns
387387
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -412,7 +412,7 @@ jobs:
412412
uses: ./.github/actions/testagent/logs
413413
with:
414414
suffix: plugins-${{ github.job }}
415-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
415+
- uses: ./.github/actions/coverage
416416
with:
417417
flags: apm-integrations-elasticsearch
418418
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -561,7 +561,7 @@ jobs:
561561
uses: ./.github/actions/testagent/logs
562562
with:
563563
suffix: plugins-${{ github.job }}-${{ matrix.node-version }}
564-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
564+
- uses: ./.github/actions/coverage
565565
with:
566566
flags: apm-integrations-http-${{ matrix.node-version }}
567567
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -590,7 +590,7 @@ jobs:
590590
uses: ./.github/actions/testagent/logs
591591
with:
592592
suffix: plugins-${{ github.job }}
593-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
593+
- uses: ./.github/actions/coverage
594594
with:
595595
flags: apm-integrations-http2
596596
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -642,7 +642,7 @@ jobs:
642642
uses: ./.github/actions/testagent/logs
643643
with:
644644
suffix: plugins-${{ github.job }}-${{ matrix.node-version }}
645-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
645+
- uses: ./.github/actions/coverage
646646
with:
647647
flags: apm-integrations-kafkajs-${{ matrix.node-version }}
648648
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -884,7 +884,7 @@ jobs:
884884
uses: ./.github/actions/testagent/logs
885885
with:
886886
suffix: plugins-${{ github.job }}
887-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
887+
- uses: ./.github/actions/coverage
888888
with:
889889
flags: apm-integrations-net
890890
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -942,7 +942,7 @@ jobs:
942942
uses: ./.github/actions/testagent/logs
943943
with:
944944
suffix: plugins-${{ github.job }}-${{ matrix.version }}-${{ matrix.range_clean }}
945-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
945+
- uses: ./.github/actions/coverage
946946
with:
947947
flags: apm-integrations-next-${{ matrix.range_clean }}
948948
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -980,7 +980,6 @@ jobs:
980980
with:
981981
dd_api_key: ${{ secrets.DD_API_KEY }}
982982

983-
# TODO: Figure out why nyc stopped working with EACCESS errors.
984983
oracledb:
985984
runs-on: ubuntu-latest
986985
services:
@@ -1021,8 +1020,8 @@ jobs:
10211020
sudo ldconfig
10221021
- run: yarn config set ignore-engines true
10231022
- run: yarn services --ignore-engines
1024-
- run: yarn test:plugins --ignore-engines
1025-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
1023+
- run: yarn test:plugins:ci --ignore-engines
1024+
- uses: ./.github/actions/coverage
10261025
with:
10271026
flags: apm-integrations-oracledb
10281027
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -1049,7 +1048,7 @@ jobs:
10491048
uses: ./.github/actions/testagent/logs
10501049
with:
10511050
suffix: plugins-${{ github.job }}
1052-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
1051+
- uses: ./.github/actions/coverage
10531052
with:
10541053
flags: apm-integrations-pino
10551054
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -1112,7 +1111,7 @@ jobs:
11121111
uses: ./.github/actions/testagent/logs
11131112
with:
11141113
suffix: plugins-${{ github.job }}-${{ matrix.node-version }}-${{ matrix.range_clean }}
1115-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
1114+
- uses: ./.github/actions/coverage
11161115
with:
11171116
flags: apm-integrations-prisma-${{ matrix.node-version }}-${{ matrix.range_clean }}
11181117
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -1269,7 +1268,7 @@ jobs:
12691268
uses: ./.github/actions/testagent/logs
12701269
with:
12711270
suffix: plugins-${{ github.job }}
1272-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
1271+
- uses: ./.github/actions/coverage
12731272
with:
12741273
flags: apm-integrations-sharedb
12751274
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1
@@ -1303,7 +1302,7 @@ jobs:
13031302
uses: ./.github/actions/testagent/logs
13041303
with:
13051304
suffix: plugins-${{ github.job }}
1306-
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
1305+
- uses: ./.github/actions/coverage
13071306
with:
13081307
flags: apm-integrations-tedious
13091308
- uses: DataDog/junit-upload-github-action@055560f63c405095e9228ba443eee7987e22bb94 # v2.1.1

0 commit comments

Comments
 (0)