Skip to content

Commit 689c5c3

Browse files
authored
fix(router): mask email addresses in URLs (#461)
Every URL redacted for the panel and agents, and the Analog server call log, now shows an email address as [email], also when the @ is encoded as %40. Known secrets are replaced before the mask so an address can't leave part of one readable. The email-in-url lint still fires on masked URLs. Refs #372
1 parent 9880bd3 commit 689c5c3

7 files changed

Lines changed: 221 additions & 13 deletions

File tree

‎apps/docs/src/content/security.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -289,12 +289,19 @@ Some credentials are hidden wherever they appear in a string, whatever the key i
289289

290290
The URL user stays visible, so you can tell which account a call uses. Ordinary words are left alone: `Basic plan`, `Token expired` and `Digest of the week` don't match.
291291

292+
### Email addresses in URLs
293+
294+
An email address in a URL is replaced with `[email]`, in a path segment, a query value, a matrix param or the fragment, also when its `@` is encoded as `%40`. So `/invite/[email protected]?to=bob%40example.org` shows as `/invite/[email]?to=[email]`. This holds for every URL that is redacted like router URLs: router navigations and link targets, SSR & HTTP request and page URLs, TransferState request URLs, the NgRx page URL, `httpResource` request URLs, Capacitor and Analog URLs, and the URLs agent tools answer with.
295+
296+
An `@` that isn't part of an address stays, as in `/@angular/core`, `[email protected]` or `/users/@alice`. The user in a URL's authority, such as `git` in `ssh://[email protected]`, stays visible like the user before a URL password. Inside an encoded URL in a query value (`?next=https%3A%2F%2Fbob%40example.com`) there is no authority to tell apart, so the address is masked, host included. The router lint (`lint-routes` and the [Lint](./inspectors/router.md#lint) view) still reports the `email-in-url` rule for a navigation whose address was masked.
297+
292298
### Router
293299

294300
These are replaced with `[redacted]` in URLs, params, data and messages:
295301

296302
- query, matrix and fragment keys that look secret (token, password, api key, code, code verifier, sig, session, jwt and similar), including inside encoded return URLs,
297303
- URL passwords, `Authorization` credentials, JWTs, bearer tokens and long opaque tokens,
304+
- email addresses, which show as `[email]` (see [Email addresses in URLs](#email-addresses-in-urls)),
298305
- route params with secret-looking names.
299306

300307
A secret route param is known from the route config before a navigation is recognized, so link targets and a navigation that was already running when the overlay attached are masked too. This covers routes in every outlet, so the token in `/(side:reset/:token)` is masked like the one in `/reset/:token`. A navigation that fails before that (for example inside a lazy route that failed to load) can still show it in its URL. JWTs are masked before long values are cut, so a long token never leaves a readable start behind.

‎packages/devtools/CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,7 @@ What remains: `/config` exports `DEFAULT_POPUP_HOTKEY`, `PANGULAR_ACTIONS`, `PAN
108108
- Keep trusted session tokens in a `0600` file per project, and expire them after 7 days.
109109
- Stop `server.allowedHosts: true` from turning off the origin checks, and warn about it.
110110
- Mask URL passwords and `Authorization` credentials (`Basic`, `Digest`, `Token`) in everything sent to the panel and to agents.
111+
- Mask email addresses in URLs as `[email]`, also with the `@` encoded as `%40`, in every redacted URL (router, HTTP, SSR, TransferState and the rest) and in the Analog server call log. An `@` that isn't an address, such as `/@angular/core`, and the user in a URL's authority stay (#372).
111112
- Mask more secrets: OpenAI, Anthropic, GitLab, npm, Hugging Face and SendGrid keys, Slack webhook URLs, the `Ocp-Apim-Subscription-Key` header, secret values that run past separators, secret XML elements and attributes, and token-like positional parts of TanStack Query keys and mutation variables.
112113
- Mark page text as untrusted in every agent tool answer.
113114
- Refuse writes when the target tab is unclear, instead of acting on whichever tab changed last.

‎packages/devtools/src/__tests__/credential-redaction.test.ts‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,12 +48,14 @@ describe('URL passwords', () => {
4848
'ssh://[email protected]/org/repo.git',
4949
'https://example.com/users/@alice',
5050
'http://localhost:4200/a:b@c',
51-
5251
'https://example.com:8443/path',
5352
]) {
5453
expect(redactUrl(text), text).toBe(text);
5554
expect(redactMessage(text), text).toBe(text);
5655
}
56+
// An address is masked in URLs (`[email]`) but left alone in messages.
57+
expect(redactUrl('mailto:[email protected]')).toBe('mailto:[email]');
58+
expect(redactMessage('mailto:[email protected]')).toBe('mailto:[email protected]');
5759
});
5860
});
5961

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
import { EventEmitter } from 'node:events';
2+
import { afterEach, describe, expect, it } from 'vitest';
3+
import { analogMiddleware, clearCalls, recentCalls } from '../analog-server-log.ts';
4+
import type { HttpCall } from '../http-rules.ts';
5+
import { redactCall } from '../http-redact.ts';
6+
import { applyRouterEvent, redactUrl, type NavigationRecord } from '../router.ts';
7+
import { explainNavigationText, type RouterPage } from '../rpc/router-tools.ts';
8+
import { listHttpCallsText } from '../rpc/http-tools.ts';
9+
import { lintRoutes } from '../rpc/router-config-tools.ts';
10+
import { maskEmails, redactText } from '../text-redact.ts';
11+
12+
describe('emails in URLs', () => {
13+
it('masks an address in a path segment, a query value and a fragment', () => {
14+
expect(redactUrl('/invite/[email protected]')).toBe('/invite/[email]');
15+
expect(redactUrl('/invite/[email protected]/accept')).toBe('/invite/[email]/accept');
16+
expect(redactUrl('/[email protected]&page=2')).toBe(
17+
'/users?email=[email]&page=2',
18+
);
19+
expect(redactUrl('/users#[email protected]')).toBe('/users#[email]');
20+
expect(redactUrl('/users;[email protected]')).toBe('/users;contact=[email]');
21+
expect(redactUrl('https://app.example.com/u/[email protected]?x=1')).toBe(
22+
'https://app.example.com/u/[email]?x=1',
23+
);
24+
});
25+
26+
it('masks an address whose @ is percent-encoded', () => {
27+
expect(redactUrl('/invite/ada%40example.com')).toBe('/invite/[email]');
28+
expect(redactUrl('/search?q=ada%2Bnews%40example.com')).toBe('/search?q=[email]');
29+
expect(redactUrl('/login?next=%2Finvite%2Fada%40example.com')).toBe(
30+
'/login?next=%2Finvite%2F[email]',
31+
);
32+
expect(redactUrl('mailto:[email protected]')).toBe('mailto:[email]');
33+
});
34+
35+
it('masks every address and keeps the text around them', () => {
36+
expect(redactUrl('/a/[email protected]/b/[email protected]?to=eve%40example.net#x')).toBe(
37+
'/a/[email]/b/[email]?to=[email]#x',
38+
);
39+
});
40+
41+
it('leaves an @ that is not part of an address alone', () => {
42+
for (const url of [
43+
'/@angular/core',
44+
'/npm/@scope/[email protected]/index.js',
45+
'https://cdn.example.com/npm/[email protected]/lodash.min.js',
46+
'https://example.com/users/@alice',
47+
'http://localhost:4200/a:b@c',
48+
'/%40fs/etc/passwd',
49+
'/users/ada@localhost',
50+
'ssh://[email protected]/org/repo.git',
51+
'https://[email protected]/path',
52+
]) {
53+
expect(redactUrl(url), url).toBe(url);
54+
}
55+
});
56+
57+
it('still hides a URL password and keeps the host', () => {
58+
expect(redactUrl('https://ada:[email protected]/u/[email protected]')).toBe(
59+
'https://ada:[redacted]@api.example.com/u/[email]',
60+
);
61+
});
62+
63+
it('hides a known secret whole when an address overlaps its start', () => {
64+
const out = redactText('/files/[email protected]/raw', ['bob.comXYZ123'], 'url');
65+
expect(out).not.toContain('XYZ');
66+
expect(out).not.toContain('123');
67+
expect(out).toContain('[redacted]');
68+
});
69+
70+
it('stays fast on long text with many @ and %40 signs', () => {
71+
const nasty = [
72+
'a%40'.repeat(50_000),
73+
'a@'.repeat(50_000),
74+
`${'a.'.repeat(50_000)}@`,
75+
`x@${'a.'.repeat(50_000)}1`,
76+
'%2B'.repeat(50_000),
77+
];
78+
const start = performance.now();
79+
for (const text of nasty) maskEmails(text);
80+
expect(performance.now() - start).toBeLessThan(500);
81+
});
82+
});
83+
84+
describe('emails in URLs shown to the panel and to agents', () => {
85+
it('masks them in a recorded HTTP call and in list-http-calls', () => {
86+
const call: HttpCall = {
87+
id: 'c1',
88+
url: '/api/users/[email protected]?to=bob%40example.org',
89+
pageUrl: '/profile/[email protected]',
90+
redirectedTo: '/api/users/ada%40example.com/v2',
91+
method: 'GET',
92+
status: 200,
93+
durationMs: 3,
94+
side: 'client',
95+
cacheHit: false,
96+
faulted: false,
97+
at: 1_000,
98+
};
99+
const safe = redactCall(call);
100+
expect(safe.url).toBe('/api/users/[email]?to=[email]');
101+
expect(safe.pageUrl).toBe('/profile/[email]');
102+
expect(safe.redirectedTo).toBe('/api/users/[email]/v2');
103+
const text = listHttpCallsText({ serverCalls: [call], pages: [] }, {}, 2_000);
104+
expect(text).toContain('/api/users/[email]');
105+
expect(text).not.toContain('example');
106+
});
107+
108+
it('masks them in a router navigation and in explain-navigation', () => {
109+
const navigations: NavigationRecord[] = [];
110+
applyRouterEvent(navigations, { type: 0, id: 1, url: '/invite/[email protected]' }, 0);
111+
applyRouterEvent(
112+
navigations,
113+
{ type: 1, id: 1, urlAfterRedirects: '/invite/ada%40example.com/welcome' },
114+
5,
115+
);
116+
expect(navigations[0]).toMatchObject({
117+
url: '/invite/[email]',
118+
finalUrl: '/invite/[email]/welcome',
119+
});
120+
const page: RouterPage = {
121+
pageId: 'aaaa',
122+
snapshot: null,
123+
navigations,
124+
reportedAt: 1_000,
125+
changedAt: 1_000,
126+
} as unknown as RouterPage;
127+
const text = explainNavigationText({ pages: [page] }, {}, 1_000);
128+
expect(text).toContain('/invite/[email]');
129+
expect(text).not.toContain('example');
130+
});
131+
132+
it('still flags the email-in-url lint on a navigation the page already masked', () => {
133+
const page = {
134+
pageId: 'aaaa',
135+
config: [],
136+
navigations: [{ id: 1, url: redactUrl('/invite/[email protected]'), trigger: 'imperative' }],
137+
} as unknown as Parameters<typeof lintRoutes>[0];
138+
const finding = lintRoutes(page).find((f) => f.rule === 'email-in-url');
139+
expect(finding?.route).toBe('/invite/[email]');
140+
});
141+
});
142+
143+
class FakeRes extends EventEmitter {
144+
statusCode = 200;
145+
headers: Record<string, string> = {};
146+
getHeader(name: string) {
147+
return this.headers[name.toLowerCase()];
148+
}
149+
write() {
150+
return true;
151+
}
152+
end() {
153+
this.emit('finish');
154+
return this;
155+
}
156+
}
157+
158+
describe('emails in Analog server call URLs', () => {
159+
afterEach(() => clearCalls());
160+
161+
it('masks them in the call URL, route and redirect location', () => {
162+
const req = {
163+
url: '/index.html',
164+
originalUrl: '/api/_analog/pages/invite/[email protected]?to=bob%40example.org',
165+
method: 'POST',
166+
headers: {},
167+
} as never;
168+
const res = new FakeRes();
169+
res.statusCode = 303;
170+
res.headers['location'] = '/welcome/[email protected]';
171+
analogMiddleware('api')(req, res as never, () => undefined);
172+
res.end();
173+
const call = recentCalls().at(-1)!;
174+
expect(call.url).toBe('/api/_analog/pages/invite/[email]?to=[email]');
175+
expect(call.route).toBe('/invite/[email]');
176+
expect(call.location).toBe('/welcome/[email]');
177+
});
178+
});

‎packages/devtools/src/analog-server-log.ts‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ import type { IncomingMessage, ServerResponse } from 'node:http';
22
import { redactCredentials } from './credential-redact.ts';
33
import { isRedactedKey, redactJwt } from './forms-privacy.ts';
44
import { redactJsonText } from './json-text-redact.ts';
5+
import { maskEmails } from './text-redact.ts';
56

67
const BEARER = /\bBearer\s+[\w.~+/=-]+/gi;
78
const QUERY_PAIR = /([?&])([^=&#\s"'\\]*)=([^&#\s"'\\]*)/g;
@@ -250,8 +251,8 @@ export function analogMiddleware(apiPrefix = 'api') {
250251
at: Date.now(),
251252
kind: match.kind,
252253
method: req.method ?? 'GET',
253-
url: redactMessage(url),
254-
route: match.route,
254+
url: maskEmails(redactMessage(url)),
255+
route: match.route && maskEmails(match.route),
255256
status: res.statusCode,
256257
ms: Math.round(performance.now() - start),
257258
bytes,
@@ -278,7 +279,7 @@ export function analogMiddleware(apiPrefix = 'api') {
278279
call.outcome = actionOutcome(res.statusCode, !!res.getHeader('x-analog-errors'));
279280
const location = res.getHeader('location');
280281
if (call.outcome === 'redirect' && location)
281-
call.location = redactMessage(String(location));
282+
call.location = maskEmails(redactMessage(String(location)));
282283
}
283284
const preview = previewOf(
284285
Buffer.concat(chunks).toString('utf8'),

‎packages/devtools/src/rpc/router-config-tools.ts‎

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import type { RouteNode } from '../router-config.ts';
22
import type { NavigationRecord } from '../router.ts';
33
import type { ServerRouteEntry } from './server-routes.ts';
4+
import { hasEmail, maskEmails } from '../text-redact.ts';
45
import { code, UNTRUSTED } from './forms-tools.ts';
56
import {
67
capped,
@@ -45,13 +46,6 @@ export interface LintFinding {
4546
}
4647

4748
const MAX_LIST = 400;
48-
const EMAIL_IN_URL = /[^/?&=]+@[^/?&=]+\.[a-z]{2,}/i;
49-
50-
/** `url` with each email address in it, plain or with `@` encoded as
51-
* `%40`, replaced by `[email]`. */
52-
function maskEmails(url: string): string {
53-
return url.replace(/[^/?&=]+(?:@|%40)[^/?&=]+\.[a-z]{2,}/gi, '[email]');
54-
}
5549

5650
function walk(
5751
nodes: RouteNode[] | undefined,
@@ -544,7 +538,7 @@ export function lintRoutes(
544538
});
545539
}
546540
}
547-
const emailed = page.navigations.find((nav) => EMAIL_IN_URL.test(nav.url));
541+
const emailed = page.navigations.find((nav) => hasEmail(nav.url));
548542
if (emailed) {
549543
findings.push({
550544
rule: 'email-in-url',

‎packages/devtools/src/text-redact.ts‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,30 @@ const STRONG_PAIRS = new Set(['apikey', 'privatekey', 'secretkey', 'accesskey'])
4848
const URL_PAIR = /(^|[?&#;])((?:(?!%3D)[^=&#;?\s/])+)(=|%3D)([^&#;\s]*)/gi;
4949
const MAX_KNOWN_SECRETS = 100;
5050

51+
/** What an email address in a URL becomes. */
52+
export const EMAIL_MASK = '[email]';
53+
/**
54+
* An email address in a URL, with its `@` plain or encoded as `%40`. The local
55+
* part only starts after a separator or a percent escape (not inside a run of
56+
* address characters) and holds no `@` or `%40`, so each run is tried once and
57+
* the scan stays linear. An optional leading `://` captures a URL's authority
58+
* (`ssh://[email protected]`), which is left alone: its password is hidden by the
59+
* credential pattern and its host stays readable.
60+
*/
61+
const EMAIL_IN_URL =
62+
/((?:^|:)\/\/)?(?<=^|[^\w.+%-]|%(?!2b)[\da-f]{2})(?:[\w.+-]|%2b)+(?:@|%40)[a-z\d-]+(?:\.[a-z\d-]+)*\.[a-z]{2,}/gi;
63+
64+
/** `url` with each email address in it (`[email protected]`, `a%40b.com`) replaced by `[email]`. */
65+
export function maskEmails(url: string): string {
66+
if (!url.includes('@') && !url.includes('%40')) return url;
67+
return url.replace(EMAIL_IN_URL, (match, authority?: string) => (authority ? match : EMAIL_MASK));
68+
}
69+
70+
/** Whether `url` holds an email address or one `maskEmails` already masked. */
71+
export function hasEmail(url: string): boolean {
72+
return url.includes(EMAIL_MASK) || url.includes('%5Bemail%5D') || maskEmails(url) !== url;
73+
}
74+
5175
/** Secret values seen this session, such as secret route params, newest last. */
5276
export const knownSecrets = new Set<string>();
5377

@@ -241,5 +265,6 @@ export function redactText(
241265
out = out.split(form).join(REDACTED);
242266
}
243267
}
244-
return out;
268+
// After the known secrets, so an address can't swallow the start of one and leave the rest readable.
269+
return mode === 'url' ? maskEmails(out) : out;
245270
}

0 commit comments

Comments
 (0)