Skip to content

feat(asm): add normalized HTTP route span tag for Tornado#18398

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits into
mainfrom
christophe-papazian/appsec-normalized-route-tornado
Jun 4, 2026
Merged

feat(asm): add normalized HTTP route span tag for Tornado#18398
gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits into
mainfrom
christophe-papazian/appsec-normalized-route-tornado

Conversation

@christophe-papazian

@christophe-papazian christophe-papazian commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

APPSEC-65478

Description

Extends RFC-1103 _dd.appsec.normalized_route to Tornado, following the FastAPI/Starlette (#17920), Django (#18209), and Flask (#18343) implementations.

Tornado's ddtrace integration produces http.route strings with %s as a placeholder for every capturing group (via _regex_to_route). The new normalizer maps those placeholders to parameter names sourced from path_params: a dict for named groups ((?P<name>...)), a list for positional groups, or {} for static routes. Optional trailing-slash patterns (/?) are treated as not declaring a trailing slash per RFC-1103 rule 1, consistent with the Django ^asm/?$/asm convention.

Changes:

  • _normalized_route.py — adds _normalize_route_tornado_cached (LRU-cached, keyed on route + param-names tuple) and normalize_route_tornado (public entry point). Handles %s placeholder mapping, /? stripping, multi-param-in-segment combining with + (rule 5), and static-segment URL-encoding (rule 3).
  • _handlers.py — registers "tornado": normalize_route_tornado in _NORMALIZED_ROUTE_BY_INTEGRATION.
  • tornado_app/app.py — converts the /asm/ route from positional to named capturing groups so path_params arrives as a dict with proper parameter names; adds MultiParamHandler (/multi-param/) and FilesHandler (/files/) for integration test coverage.
  • test_tornado.py — overrides test_normalized_route and test_normalized_route_disabled_when_api_security_off with Tornado-specific expected values (no trailing slash for /? routes); expands ENDPOINT_DISCOVERY_EXPECTED_PATHS.
  • test_normalized_route.py — 27 new unit tests for normalize_route_tornado covering all code paths.
  • utils.py — documents why Tornado is skipped in test_normalized_route_survives_request_span_name_override (span name is hard-coded via schematize_url_operation).

Testing

  • 27 unit tests in tests/appsec/appsec/api_security/test_normalized_route.py covering named groups, positional groups, multi-param combining (+), /? stripping, explicit trailing-slash preservation, URL-encoding, mismatch detection, and lru_cache identity.
  • Integration tests in test_tornado.py exercise the full request path: named-group params (/asm/{param_int}/{param_str}), multi-param-in-segment (/multi-param/{first+last}), catch-all (/files/{file_path}), root (/), ASM-disabled gate, and API-Security-disabled gate.
  • test_normalized_route_disabled_when_api_security_off verifies the tag is absent when API Security is off while ASM is on.

Risks

None — all production code is confined to ddtrace/appsec/. No changes to integration code or public API. The /asm/ route change (positional → named groups) is backward-compatible: Tornado passes named groups as keyword arguments that match the existing handler signature.

Additional Notes

Tornado routes using positional groups (no (?P<name>...)) produce auto-numbered placeholders ({param1}, {param2}, …). Users who want named parameters in _dd.appsec.normalized_route should use named capturing groups in their route patterns.

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Pipelines  Tests

Fix all issues with BitsAI

⚠️ Warnings

🚦 8 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-py | build linux serverless: [amd64, cp315-cp315, v113741238-d2b8243-manylinux2014_x86_64, 1]   View in Datadog   GitLab

See error Error during artifact generation. Missing required tool or configuration for generating manylinux wheels.

DataDog/apm-reliability/dd-trace-py | build linux serverless: [amd64, cp315-cp315, v113741491-d2b8243-musllinux_1_2_x86_64, 1]   View in Datadog   GitLab

See error NotImplementedError: This version of CPython is not supported yet

DataDog/apm-reliability/dd-trace-py | build linux serverless: [arm64, cp315-cp315, v113741357-d2b8243-manylinux2014_aarch64, 1]   View in Datadog   GitLab

See error ImportError: NotImplementedError: This version of CPython is not supported yet during wheel testing.

View all 8 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 6ab0ab1 | Docs | Datadog PR Page | Give us feedback!

@christophe-papazian christophe-papazian added the ASM Application Security Monitoring label Jun 2, 2026
@christophe-papazian
christophe-papazian force-pushed the christophe-papazian/appsec-normalized-route-tornado branch from 8928751 to b86f35d Compare June 2, 2026 09:05
@pr-commenter

pr-commenter Bot commented Jun 2, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-06-04 12:20:39

Comparing candidate commit 6ab0ab1 in PR branch christophe-papazian/appsec-normalized-route-tornado with baseline commit e9662f3 in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 83 metrics, 0 unstable metrics.

scenario:iastaspectsospath-ospathbasename_aspect

  • 🟥 execution_time [+96.833µs; +104.254µs] or [+23.023%; +24.787%]

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jun 2, 2026

Copy link
Copy Markdown

Codeowners resolved as

ddtrace/appsec/_api_security/_normalized_route.py                       @DataDog/asm-python
ddtrace/appsec/_handlers.py                                             @DataDog/asm-python
releasenotes/notes/appsec-normalized-route-tornado-0c8638ac0f2529c9.yaml  @DataDog/apm-python
tests/appsec/appsec/api_security/test_normalized_route.py               @DataDog/asm-python
tests/appsec/contrib_appsec/test_tornado.py                             @DataDog/asm-python
tests/appsec/contrib_appsec/tornado_app/app.py                          @DataDog/asm-python
tests/appsec/contrib_appsec/utils.py                                    @DataDog/asm-python

@christophe-papazian
christophe-papazian force-pushed the christophe-papazian/appsec-normalized-route-tornado branch 6 times, most recently from 8c999a6 to cef37b4 Compare June 3, 2026 08:46
@christophe-papazian
christophe-papazian marked this pull request as ready for review June 3, 2026 09:46
@christophe-papazian
christophe-papazian requested review from a team as code owners June 3, 2026 09:46

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cef37b41c8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ddtrace/appsec/_api_security/_normalized_route.py Outdated
@christophe-papazian
christophe-papazian marked this pull request as draft June 3, 2026 10:08
@christophe-papazian
christophe-papazian force-pushed the christophe-papazian/appsec-normalized-route-tornado branch 4 times, most recently from 6ae8f9d to 37459ab Compare June 3, 2026 12:01
@christophe-papazian
christophe-papazian marked this pull request as ready for review June 3, 2026 13:29
@christophe-papazian

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 37459ab39e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ddtrace/appsec/_api_security/_normalized_route.py Outdated
@christophe-papazian
christophe-papazian force-pushed the christophe-papazian/appsec-normalized-route-tornado branch from 37459ab to 3b721b8 Compare June 3, 2026 13:54
@christophe-papazian

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3b721b8c0c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ddtrace/appsec/_api_security/_normalized_route.py

@emmettbutler emmettbutler left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

release note approved

Comment thread releasenotes/notes/appsec-normalized-route-tornado-0c8638ac0f2529c9.yaml Outdated
Implements RFC-1103 ``_dd.appsec.normalized_route`` for Tornado, following
the FastAPI/Starlette and Django implementations already in main.

- ``normalize_route_tornado`` in ``_api_security/_normalized_route.py``:
  handles Tornado's ``%s``-placeholder ``http.route`` strings (produced by
  ``_regex_to_route``), named-group dict and positional-group list
  ``path_params``, multi-param segment combining (rule 5 ``+``), and
  optional-trailing-slash stripping (``/?`` is not "declared with trailing
  slash" per RFC rule 1).
- ``_handlers.py``: registers ``"tornado"`` in
  ``_NORMALIZED_ROUTE_BY_INTEGRATION``.
- ``tornado_app/app.py``: switches ``/asm/`` route to named groups so
  ``path_params`` arrives as a dict with proper names; adds
  ``MultiParamHandler`` (``/multi-param/``) and ``FilesHandler``
  (``/files/``) for normalized-route integration tests.
- ``test_tornado.py``: overrides ``test_normalized_route`` and
  ``test_normalized_route_disabled_when_api_security_off`` with
  Tornado-specific expected values; expands
  ``ENDPOINT_DISCOVERY_EXPECTED_PATHS``.
- ``test_normalized_route.py``: adds 27 unit test cases covering all
  code paths of ``normalize_route_tornado``.
- ``utils.py``: documents the reason Tornado is skipped in
  ``test_normalized_route_survives_request_span_name_override``.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
@christophe-papazian
christophe-papazian force-pushed the christophe-papazian/appsec-normalized-route-tornado branch from 3b721b8 to 4c95976 Compare June 4, 2026 09:32
@christophe-papazian
christophe-papazian force-pushed the christophe-papazian/appsec-normalized-route-tornado branch from 408222d to 302da8c Compare June 4, 2026 11:05
@christophe-papazian
christophe-papazian force-pushed the christophe-papazian/appsec-normalized-route-tornado branch from 302da8c to 6ab0ab1 Compare June 4, 2026 11:58
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 79d1e67 into main Jun 4, 2026
632 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the christophe-papazian/appsec-normalized-route-tornado branch June 4, 2026 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ASM Application Security Monitoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants