Skip to content

chore: add telemetry checks to AAS web app e2e tests#2365

Merged
ava-silver merged 10 commits into
masterfrom
ava.silver/chore/add-web-app-e2e-checks
Jun 22, 2026
Merged

chore: add telemetry checks to AAS web app e2e tests#2365
ava-silver merged 10 commits into
masterfrom
ava.silver/chore/add-web-app-e2e-checks

Conversation

@ava-silver

@ava-silver ava-silver commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

What and why?

Adds telemetry-flow verification to the Azure App Service (Linux + Windows) e2e suites so they assert that telemetry actually reaches Datadog under a run-specific identity -- not just that instrumentation config was applied. Linux asserts spans and logs; Windows asserts spans (Windows App Service does not support Datadog log collection).

Stacked on #2364, and depends on #2369 (shared telemetry checker).

How?

  • Instrument with explicit service / env / version and a one_e2e_run_id extra tag so the checker asserts identity, not mere existence.
  • Instrument with the same Datadog API key the telemetry checker queries with (DATADOG_API_KEY), so the spans/logs land in the org the assertions read from.
  • Linux: pass --instance-logging so the sidecar forwards the app's stdout logs, not just traces.
  • Windows: deploy the app via az webapp deploy (zipdeploy) with SCM_DO_BUILD_DURING_DEPLOYMENT, so Oryx generates the iisnode web.config IIS needs to route requests to the Node app. (Mounting the package read-only via WEBSITE_RUN_FROM_PACKAGE skips that build, so the app 403s.) Auth uses the az-login AAD token; the SCM site can return a 502 while cold, so the deploy retries. (Requires the CI service principal to have Website Contributor on the e2e resource group for the SCM deploy.)
  • Drive sustained traffic via the shared triggerTraffic helper so the trace/log pipeline has time to warm up after a cold start before we assert telemetry.
  • telemetry flows asserts identity (service + env + version + run-id tag) via the shared telemetry checker -- spans + logs on Linux, spans only on Windows.
  • Add an idempotency test: re-instrument is a no-op and config stays singular.
  • Add the shared traffic.ts and telemetry checker to the AAS CI path filter.

Test plan

  • AAS Linux + Windows e2e run in the End-to-end test the package GitHub Actions job (gated by the aas path filter + Azure OIDC); all three Node matrix versions green with live spans (Linux: spans + logs) flowing to Datadog.

Review checklist

  • Feature or bugfix MUST have appropriate tests (unit, integration)

ava-silver commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jun 16, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 64bd964 | Docs | Datadog PR Page | Give us feedback!

@ava-silver ava-silver added the chores Related to the CI or developer experience label Jun 16, 2026
@ava-silver
ava-silver changed the base branch from master to graphite-base/2365 June 16, 2026 18:15
@ava-silver
ava-silver force-pushed the graphite-base/2365 branch from 36742bd to 4aa83a2 Compare June 16, 2026 18:15
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-web-app-e2e-checks branch from 84fd1d0 to 0f29354 Compare June 16, 2026 18:15
@ava-silver
ava-silver changed the base branch from graphite-base/2365 to ava.silver/chore/add-container-app-e2e-checks June 16, 2026 18:15
@ava-silver
ava-silver changed the base branch from ava.silver/chore/add-container-app-e2e-checks to graphite-base/2365 June 16, 2026 18:23
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-web-app-e2e-checks branch from 0f29354 to 410a930 Compare June 16, 2026 18:24
@ava-silver
ava-silver force-pushed the graphite-base/2365 branch from 4aa83a2 to 1a359ff Compare June 16, 2026 18:24
@ava-silver
ava-silver changed the base branch from graphite-base/2365 to ava.silver/chore/add-container-app-e2e-checks June 16, 2026 18:24
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-web-app-e2e-checks branch 3 times, most recently from 410a930 to b4027cc Compare June 17, 2026 17:52
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-container-app-e2e-checks branch from 2bfcdae to c30c9df Compare June 17, 2026 17:54
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-web-app-e2e-checks branch 2 times, most recently from 4bb9bc5 to 9e1091d Compare June 17, 2026 18:23
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-container-app-e2e-checks branch from c30c9df to 6cd5dc1 Compare June 17, 2026 18:23
@ava-silver ava-silver added the serverless Related to [aas, cloud-run, lambda, stepfunctions] label Jun 17, 2026 — with Graphite App
@ava-silver
ava-silver marked this pull request as ready for review June 17, 2026 20:04
@ava-silver
ava-silver requested review from a team as code owners June 17, 2026 20:04
@ava-silver
ava-silver requested a review from TalUsvyatsky June 17, 2026 20:04
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-web-app-e2e-checks branch from 9e1091d to bd78eea Compare June 17, 2026 20:37
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-container-app-e2e-checks branch 2 times, most recently from 9fd84c5 to e700d40 Compare June 17, 2026 20:45
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-web-app-e2e-checks branch from 73c9fbb to dbee69c Compare June 18, 2026 22:33
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-container-app-e2e-checks branch from e700d40 to 112e548 Compare June 18, 2026 22:33
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-web-app-e2e-checks branch from dbee69c to c8512a8 Compare June 22, 2026 14:37
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-container-app-e2e-checks branch 2 times, most recently from 8cb96cd to b6cd212 Compare June 22, 2026 14:50
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-web-app-e2e-checks branch 2 times, most recently from 94cef46 to 6905508 Compare June 22, 2026 15:26
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-container-app-e2e-checks branch 2 times, most recently from 426ebf0 to a5a35eb Compare June 22, 2026 16:28
@ava-silver
ava-silver changed the base branch from ava.silver/chore/add-container-app-e2e-checks to graphite-base/2365 June 22, 2026 17:03
Uses pre-built zips from selfmonitoringprod storage (node-sidecar.zip for
Linux, node-extension.zip for Windows) -- these include dd-trace + winston
with node_modules pre-installed, so no SCM_DO_BUILD_DURING_DEPLOYMENT needed.
az webapp deploy requires Kudu/SCM permissions that the CI service principal
doesn't have. WEBSITE_RUN_FROM_PACKAGE uses the ARM appsettings API (same as
aas instrument) and mounts the zip directly -- no extra permissions needed.
@ava-silver
ava-silver force-pushed the ava.silver/chore/add-web-app-e2e-checks branch from 6905508 to 64bd964 Compare June 22, 2026 18:42
@ava-silver
ava-silver force-pushed the graphite-base/2365 branch from a5a35eb to 5b46b0c Compare June 22, 2026 18:42
@ava-silver
ava-silver changed the base branch from graphite-base/2365 to master June 22, 2026 18:42
@ava-silver
ava-silver merged commit f7a5538 into master Jun 22, 2026
35 checks passed
@ava-silver
ava-silver deleted the ava.silver/chore/add-web-app-e2e-checks branch June 22, 2026 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chores Related to the CI or developer experience serverless Related to [aas, cloud-run, lambda, stepfunctions]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants