Skip to content

RUMM-2377 Add checks on intake request headers#1005

Merged
xgouchet merged 1 commit into
developfrom
xgouchet/RUMM-2377/sanitize_upload_headers
Aug 16, 2022
Merged

RUMM-2377 Add checks on intake request headers#1005
xgouchet merged 1 commit into
developfrom
xgouchet/RUMM-2377/sanitize_upload_headers

Conversation

@xgouchet

@xgouchet xgouchet commented Aug 3, 2022

Copy link
Copy Markdown
Contributor

What does this PR do?

Sanitize our intake request headers to be complient with the HTTP specification

Motivation

Before OkHttp version 4.9.2 (cf okhttp#6551), if a header value contains illegal characters (any non ASCII or Control character), the header would be printed as is in the logcat. This could potentially leak secrets (especially our customer's client token).

This is not a critical issue since:

  • Client Tokens are generated by us and all match /pub[0-9a-f]+/, so they never have any invalid characters
  • Only the invalid header is printed to the logcat, so even if another header was invalid, it would not print the client token

Additional Notes

We are still sanitizing every header that might contain value that we do not control.

  • if the client token would lead to an error, we simply don't submit the request
  • for other headers, we remove the illegal characters. It might lead to unexpected value but most of the data would still be there.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Make sure you discussed the feature or bugfix with the maintaining team in an Issue
  • Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

@xgouchet
xgouchet requested a review from a team as a code owner August 3, 2022 12:40
@xgouchet
xgouchet requested a review from aparzio August 3, 2022 12:41
@xgouchet xgouchet added the size-small This PR is small sized label Aug 3, 2022
"Datadog/$sdkVersion " +
sanitizeHeaderValue(System.getProperty(SYSTEM_UA))
.ifBlank {
"Datadog/${sanitizeHeaderValue(sdkVersion)} " +

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not even sure if the sdkVersion should be sanitized at all

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sdkVersion will possibly be updatable from crossplatform SDKs, meaning it will be using public APIs. So in theory, it could have an invalid character yes

@codecov-commenter

codecov-commenter commented Aug 4, 2022

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.81818% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.22%. Comparing base (72beeb9) to head (ecb6433).
⚠️ Report is 3985 commits behind head on develop.

Files with missing lines Patch % Lines
.../android/core/internal/net/DataOkHttpUploaderV2.kt 81.82% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1005      +/-   ##
===========================================
+ Coverage    83.19%   83.22%   +0.03%     
===========================================
  Files          270      270              
  Lines         9209     9212       +3     
  Branches      1463     1468       +5     
===========================================
+ Hits          7661     7666       +5     
+ Misses        1137     1134       -3     
- Partials       411      412       +1     
Files with missing lines Coverage Δ
.../android/core/internal/net/DataOkHttpUploaderV2.kt 97.62% <81.82%> (-1.15%) ⬇️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@aparzio aparzio left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got a couple of questions.

}

private fun isValidHeaderValueChar(c: Char): Boolean {
return c == '\t' || c in '\u0020'..'\u007F'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of questions:

  1. isn't 7F a control char? shouldn't be excluded?
  2. are you sure that \t doesn't trigger the vuln?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

\t indeed is a valid character (all ascii whitespace chars are allowed : , \n and \t. But you're right 0x7f should be excluded.

System.getProperty(SYSTEM_UA).let {
if (it.isNullOrBlank()) {
"Datadog/$sdkVersion " +
sanitizeHeaderValue(System.getProperty(SYSTEM_UA))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curiosity: Why don't you trust SYSTEM_UA property but you trust your androidInfoProvider, which relays on Build class, which in turns take information from system properties?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly because the syntax for User Agent can become very sketchy and more likely to have weird characters.
Also the Build.xxx will be already fetched from the app zygote, whereas the UA is retrieved at Runtime, making it possible for the host app to override it.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got your point. It is a good trade-off.

@xgouchet
xgouchet force-pushed the xgouchet/RUMM-2377/sanitize_upload_headers branch from 4c85864 to ecb6433 Compare August 8, 2022 06:38
@xgouchet
xgouchet requested a review from aparzio August 8, 2022 06:50

@aparzio aparzio left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

System.getProperty(SYSTEM_UA).let {
if (it.isNullOrBlank()) {
"Datadog/$sdkVersion " +
sanitizeHeaderValue(System.getProperty(SYSTEM_UA))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got your point. It is a good trade-off.

@xgouchet
xgouchet merged commit 694b4b8 into develop Aug 16, 2022
@xgouchet
xgouchet deleted the xgouchet/RUMM-2377/sanitize_upload_headers branch August 16, 2022 08:20
@xgouchet xgouchet added this to the 1.15.0 milestone Dec 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size-small This PR is small sized

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants