Skip to content

chore: add script to catalog DD routes per scope#2343

Merged
ava-silver merged 2 commits into
masterfrom
ava.silver/chore/add-script-to-catalog-used-endpoints
Jun 9, 2026
Merged

chore: add script to catalog DD routes per scope#2343
ava-silver merged 2 commits into
masterfrom
ava.silver/chore/add-script-to-catalog-used-endpoints

Conversation

@ava-silver

@ava-silver ava-silver commented Jun 8, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

What and why?

Adds bin/catalog-endpoints.ts -- a dev script that statically analyzes the codebase and produces endpoints.csv: a mapping of CLI scopes to the Datadog API routes they call, with HTTP methods. This supports SEAL (Smart Edge Audit Logs) passive observability -- CI will upload the CSV to S3 to keep a reference table current so audit log data can be enriched with CLI command context.

Right now uses a grep-based approach, but we could switch to an AST parsing approach or something else in the future if we want more capability

Example output:

scope,route,method
cloud-run,/api/ui/support/serverless/flare,POST
coverage,/api/v2/cicovreprt,POST
deployment,/api/v2/ci/deployments/correlate,POST
deployment,/api/v2/ci/deployments/correlate-image,POST
deployment,/api/v2/deployments/gates/evaluation,POST
deployment,/api/v2/deployments/gates/evaluation/:evaluationId,GET
dora,/api/v2/dora/deployment,POST
dsyms,/api/v2/srcmap,GET
elf-symbols,/api/v2/srcmap,GET
flutter-symbols,/api/v2/srcmap,GET
gate,/api/v2/quality-gates/evaluate,POST
git-metadata,/api/v2/git/repository/packfile,GET
git-metadata,/api/v2/git/repository/search_commits,GET
git-metadata,/api/v2/srcmap,GET
junit,/api/v2/cireport,POST
lambda,/api/ui/support/serverless/flare,POST
measure,/api/v2/ci/pipeline/metrics,POST
pe-symbols,/api/v2/srcmap,GET
react-native,/api/v2/srcmap,GET
sarif,/api/v2/cicodescan,POST
sbom,/api/v2/static-analysis-sca/dependencies,POST
sourcemaps,/api/v2/srcmap,GET
synthetics,/synthetics/ci/batch/:batchId,GET
synthetics,/synthetics/ci/tunnel,GET
synthetics,/synthetics/mobile/applications/:applicationId/multipart-presigned-urls,POST
synthetics,/synthetics/mobile/applications/:applicationId/multipart-upload-complete,POST
synthetics,/synthetics/mobile/applications/validation-job-status/:jobId,GET
synthetics,/synthetics/settings,GET
synthetics,/synthetics/tests/:testId,GET
synthetics,/synthetics/tests/:testId,PUT
synthetics,/synthetics/tests/:testId/version_history/:version?only_check_existence=true,GET
synthetics,/synthetics/tests/:testType/:testId,GET
synthetics,/synthetics/tests/poll_results,GET
synthetics,/synthetics/tests/search,GET
synthetics,/synthetics/tests/trigger/ci,POST
tag,/api/v2/ci/pipeline/tags,POST
terraform,/api/v2/ciiac,POST
trace,/api/intake/ci/custom_spans,POST
unity-symbols,/api/v2/srcmap,GET

How?

  • Globs packages/plugin-*/src and packages/base/src/commands/*/ to collect scopes and their source files
  • Extracts datadogRoute('...') calls via regex; resolves HTTP method by scanning up to 5 lines before / 2 lines after each call for method: 'VERB' (handles lowercase, METHOD_POST-style constants; defaults to GET)
  • Handles helpers/serverless/flare.ts as a special case -- greps which scopes import it and attributes its routes to each importer (lambda, cloud-run)
  • Skips helpers/apikey.ts (/api/v1/validate) and helpers/upload.ts (/v1/input) as they're too cross-cutting
  • Deduplicates rows and writes endpoints.csv at the repo root (gitignored)

Run with yarn catalog-endpoints.

Review checklist

  • Feature or bugfix MUST have appropriate tests (unit, integration)

@ava-silver Graphite App

ava-silver commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@datadog-datadog-prod-us1

This comment has been minimized.

Sign in to view
@ava-silver ava-silver changed the title add script to catalog DD routes per scope chore: add script to catalog DD routes per scope Jun 8, 2026
@ava-silver ava-silver added chores Related to the CI or developer experience datadog-ci For PRs spanning multiple commands, and repo-wide changes labels Jun 8, 2026
@ava-silver
ava-silver marked this pull request as ready for review June 8, 2026 16:37
@ava-silver
ava-silver requested a review from a team as a code owner June 8, 2026 16:37
@ava-silver
ava-silver requested a review from Drarig29 June 8, 2026 16:38
GabrielAnca
GabrielAnca approved these changes Jun 9, 2026
View reviewed changes

@GabrielAnca GabrielAnca left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! One small improvement proposed

Comment thread bin/catalog-endpoints.ts Outdated
Drarig29
Drarig29 reviewed Jun 9, 2026
View reviewed changes
Comment thread bin/catalog-endpoints.ts
Comment thread bin/catalog-endpoints.ts Outdated
@ava-silver
ava-silver requested a review from Drarig29 June 9, 2026 13:52
@ava-silver
ava-silver merged commit 0899a50 into master Jun 9, 2026
32 checks passed
@ava-silver
ava-silver deleted the ava.silver/chore/add-script-to-catalog-used-endpoints branch June 9, 2026 14:44
@GabrielAnca GabrielAnca mentioned this pull request Jun 17, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@GabrielAnca GabrielAnca GabrielAnca approved these changes
@Drarig29 Drarig29 Drarig29 approved these changes
@duncanista duncanista duncanista approved these changes

Assignees

No one assigned

Labels

chores Related to the CI or developer experience datadog-ci For PRs spanning multiple commands, and repo-wide changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ava-silver @GabrielAnca @Drarig29 @duncanista