Skip to content

Commit 7f3b817

Browse files
committed
fix(overlay): close redaction leaks found by a leak corpus
Add a corpus of fake secrets in many shapes (JWTs, auth schemes, key prefixes, PEM keys, DSNs, values under secret names) and run it through the redaction path of every collector and of the server: serialize, NgRx, router, forms, errors, HTTP calls, previews and headers, Analog reports and server logs, and Capacitor calls. It found five leaks, fixed here: - A secret pair after another word and a colon (`Error: password: "x"`, `Error: password="x"`) kept the password, because `password:` was read as the value of `Error`. text-redact.ts now reads such a value again as a key. `xmlns:auth="urn:a"` and prose like `token: expired` stay as they were. - In an Analog server body that is not JSON, `Authorization: ApiKey <key>` kept the key: the pair rule masked `ApiKey` first and left the key for the credential rule to miss. Credentials, JWTs and bearer tokens are now masked before the pairs (redactTokens in forms-privacy.ts, which the forms messages use too). - The Analog server log ignored secret pairs in the `#fragment` and in `;matrix` params. - Headers with a camelCase secret name (`newPassword`) were not masked, since the name was only checked after lowercasing (http-redact.ts). - Tokens used as NgRx state keys were not masked, though security.md says they are (ngrx-shared.ts). Two keys that mask to the same text both stay, the second with a ` (2)` suffix. Refs #371
1 parent b78a59b commit 7f3b817

9 files changed

Lines changed: 467 additions & 18 deletions

File tree

‎packages/devtools/CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -229,6 +229,7 @@ What remains: `/config` exports `DEFAULT_POPUP_HOTKEY`, `PANGULAR_ACTIONS`, `PAN
229229
- The **Pipes** tab no longer marks two `| async` usages of a Promise, a Subject, an `asObservable()` view of one or an NgRx store select as duplicates, since their subscribers share one execution (#369). Any other source that two `| async` usages subscribe to shows as **possible duplicate subscription**, and `explain-pipe` says **Possible duplicate subscriptions**, because the page can't see `share()` or `shareReplay()` in a pipe.
230230
- Browser DevTools show the app's own line as the caller of a `console` call again. The browser entries ship source maps that list the console tap in `x_google_ignoreList`, so its wrapper frame is skipped (#369).
231231
- The Angular Native overlay gates the injector profiler like the NativeScript overlay: in a development build with no `window`, importing it defines one until Angular publishes its debug utilities, so the **Providers** lists fill (#371).
232+
- Redaction leaks found by a corpus of fake secrets run through every redaction path are closed: a secret pair after another word and a colon (`Error: password: "…"`), the key after `Authorization: ApiKey` in an Analog server body that is not JSON, secret pairs in the fragment or matrix params of an Analog server message, headers with a camelCase secret name such as `newPassword`, and tokens used as NgRx state keys.
232233

233234
### Docs
234235

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
{
2+
"_comment": "Fake secrets in the shapes the redaction rules promise to hide. `shapes` hide themselves wherever they appear; `named` hide under a secret key. `secret` is the part that must never come out. None of these is a real credential. Provider keys that secret scanners flag (Stripe, Slack webhooks, GitLab) are tested in credential-redaction.test.ts instead.",
3+
"shapes": [
4+
{
5+
"name": "JWT",
6+
"value": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkFkYSJ9.dozjgNryP4J3jVmNHl0w5N_XgL0n3I9PlFUP0THsR8U",
7+
"secret": "dozjgNryP4J3jVmNHl0w5N_XgL0n3I9PlFUP0THsR8U"
8+
},
9+
{
10+
"name": "unsigned JWT",
11+
"value": "eyJhbGciOiJub25lIn0.eyJzdWIiOiJhZGEtbG92ZWxhY2UifQ.",
12+
"secret": "eyJzdWIiOiJhZGEtbG92ZWxhY2UifQ"
13+
},
14+
{
15+
"name": "bearer token",
16+
"value": "Bearer 9f8e7d6c5b4a3f2e1d0c9b8a7f6e5d4c",
17+
"secret": "9f8e7d6c5b4a3f2e1d0c9b8a7f6e5d4c"
18+
},
19+
{
20+
"name": "Basic credentials",
21+
"value": "Basic YWRhOmNvcnJlY3Rob3JzZQ==",
22+
"secret": "YWRhOmNvcnJlY3Rob3JzZQ"
23+
},
24+
{
25+
"name": "Digest credentials",
26+
"value": "Digest username=\"ada\", response=\"6629fae49393a05397450978507c4ef1\"",
27+
"secret": "6629fae49393a05397450978507c4ef1"
28+
},
29+
{
30+
"name": "Token credentials",
31+
"value": "Token 9944b09199c62bcf9418ad846dd0e4bbdfc6ee4b",
32+
"secret": "9944b09199c62bcf9418ad846dd0e4bbdfc6ee4b"
33+
},
34+
{
35+
"name": "Authorization header",
36+
"value": "Authorization: ApiKey Zq7Lm2Pw9Xr4Tn8Vb3Kd",
37+
"secret": "Zq7Lm2Pw9Xr4Tn8Vb3Kd"
38+
},
39+
{
40+
"name": "URL password",
41+
"value": "https://ada:[email protected]/app",
42+
"secret": "Tr0ub4dorPw"
43+
},
44+
{
45+
"name": "Stripe webhook secret",
46+
"value": "whsec_MfKQ9r8GKYqrTwjUPD8ILPZIo2LaLaSw",
47+
"secret": "MfKQ9r8GKYqrTwjUPD8ILPZIo2LaLaSw"
48+
},
49+
{
50+
"name": "GitHub token",
51+
"value": "ghp_16C7e42F292c6912E7710c838347Ae178B4a",
52+
"secret": "16C7e42F292c6912E7710c838347Ae178B4a"
53+
},
54+
{
55+
"name": "GitHub fine-grained token",
56+
"value": "github_pat_11ABCDEFG0123456789_abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJK",
57+
"secret": "11ABCDEFG0123456789_abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJK"
58+
},
59+
{
60+
"name": "Slack token",
61+
"value": "xoxb-2442-1234567890-AbCdEfGhIjKlMnOpQrStUvWx",
62+
"secret": "2442-1234567890-AbCdEfGhIjKlMnOpQrStUvWx"
63+
},
64+
{
65+
"name": "AWS access key id",
66+
"value": "AKIAIOSFODNN7EXAMPLE",
67+
"secret": "IOSFODNN7EXAMPLE"
68+
},
69+
{
70+
"name": "Google API key",
71+
"value": "AIzaSyD-9tSrke72PouQMnMX-a7eZSW0jkFMBWY",
72+
"secret": "SyD-9tSrke72PouQMnMX-a7eZSW0jkFMBWY"
73+
},
74+
{
75+
"name": "OpenAI project key",
76+
"value": "sk-proj-AbCdEfGhIjKlMnOpQrStUvWxYz0123456789abcd",
77+
"secret": "AbCdEfGhIjKlMnOpQrStUvWxYz0123456789abcd"
78+
},
79+
{
80+
"name": "Anthropic key",
81+
"value": "sk-ant-api03-AbCdEfGhIjKlMnOpQrStUvWxYz0123456789-_abcd",
82+
"secret": "api03-AbCdEfGhIjKlMnOpQrStUvWxYz0123456789-_abcd"
83+
},
84+
{
85+
"name": "npm token",
86+
"value": "npm_AbCdEfGhIjKlMnOpQrStUvWxYz0123456789",
87+
"secret": "AbCdEfGhIjKlMnOpQrStUvWxYz0123456789"
88+
},
89+
{
90+
"name": "Hugging Face token",
91+
"value": "hf_AbCdEfGhIjKlMnOpQrStUvWxYz01234567",
92+
"secret": "AbCdEfGhIjKlMnOpQrStUvWxYz01234567"
93+
},
94+
{
95+
"name": "SendGrid key",
96+
"value": "SG.AbCdEfGhIjKlMnOpQrStUv.WxYz0123456789AbCdEfGhIjKlMn",
97+
"secret": "WxYz0123456789AbCdEfGhIjKlMn"
98+
},
99+
{
100+
"name": "PEM private key",
101+
"value": "-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEA7bq98fakeKeyMaterialLine\n-----END RSA PRIVATE KEY-----",
102+
"secret": "MIIEowIBAAKCAQEA7bq98fakeKeyMaterialLine"
103+
},
104+
{
105+
"name": "Sentry DSN",
106+
"value": "https://[email protected]/42",
107+
"secret": "0123456789abcdef0123456789abcdef"
108+
}
109+
],
110+
"named": [
111+
{
112+
"key": "password",
113+
"secret": "Tr0ub4dor-3"
114+
},
115+
{
116+
"key": "newPassword",
117+
"secret": "C0rrectHorse"
118+
},
119+
{
120+
"key": "apiKey",
121+
"secret": "k3y-AbC-998877"
122+
},
123+
{
124+
"key": "client_secret",
125+
"secret": "cs-Zz9Yy8Xx7"
126+
},
127+
{
128+
"key": "access_token",
129+
"secret": "at-Qq1Ww2Ee3"
130+
},
131+
{
132+
"key": "refreshToken",
133+
"secret": "rt-Pp0Oo9Ii8"
134+
},
135+
{
136+
"key": "sessionId",
137+
"secret": "sess-Mm4Nn5Bb6"
138+
},
139+
{
140+
"key": "cardNumber",
141+
"secret": "4111111111111111"
142+
},
143+
{
144+
"key": "cvv",
145+
"secret": "737-cvv"
146+
},
147+
{
148+
"key": "otp",
149+
"secret": "otp-482913"
150+
},
151+
{
152+
"key": "privateKey",
153+
"secret": "pk-Ll7Kk6Jj5"
154+
},
155+
{
156+
"key": "signature",
157+
"secret": "sig-Hh4Gg3Ff2"
158+
}
159+
]
160+
}
Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
// @vitest-environment jsdom
2+
/**
3+
* The leak corpus: fake secrets in many shapes (fixtures/redaction-corpus.json)
4+
* run through the redaction path of every collector and of the server, to
5+
* prove none of them comes out readable. A new redaction path belongs in one
6+
* of the tables below; a new secret shape belongs in the fixture.
7+
*/
8+
import { afterEach, describe, expect, it } from 'vitest';
9+
import corpus from './fixtures/redaction-corpus.json' with { type: 'json' };
10+
import {
11+
REDACTED,
12+
isRedactedKey,
13+
redactMessage,
14+
SecretSet,
15+
setRedaction,
16+
} from '../forms-privacy.ts';
17+
import { redactRecord, redactText, redactUrl } from '../router.ts';
18+
import { serialize } from '../serialize.ts';
19+
import { redactCall, redactHeaders, redactPreview } from '../http-redact.ts';
20+
import { redactJsonText } from '../json-text-redact.ts';
21+
import { redactAnalogReport } from '../analog-redact.ts';
22+
import { redactBody, redactMessage as redactServerMessage } from '../analog-server-log.ts';
23+
import { maskedArgs } from '../capacitor-redact.ts';
24+
import { serialize as ngrxSerialize } from '../ngrx-shared.ts';
25+
import { consoleText, valueText } from '../errors-collector.ts';
26+
import { serializeFormValue } from '../forms.ts';
27+
import type { AnalogRuntimeReport } from '../analog-runtime.ts';
28+
import type { HttpCall } from '../http-rules.ts';
29+
30+
afterEach(() => setRedaction());
31+
32+
type Path = (input: string) => unknown;
33+
type ObjectPath = (input: Record<string, unknown>) => unknown;
34+
35+
const call = (fields: Partial<HttpCall>) =>
36+
redactCall({ id: 1, method: 'GET', url: '/api', ...fields } as HttpCall);
37+
38+
const report = (fields: Partial<AnalogRuntimeReport>) =>
39+
redactAnalogReport({
40+
pageId: 'p1',
41+
url: '/',
42+
analog: true,
43+
chain: [],
44+
hydrated: 0,
45+
transferState: false,
46+
hydrationErrors: [],
47+
configPaths: [],
48+
...fields,
49+
});
50+
51+
const show = (value: unknown): string =>
52+
typeof value === 'string' ? value : (JSON.stringify(value) ?? '');
53+
54+
/** Paths that take free text: messages, errors, previews, values. */
55+
const PAIR_TEXT_PATHS: Record<string, Path> = {
56+
'serialize (component inputs, signals, pipes, providers)': (t) => serialize(t),
57+
'NgRx serialize': (t) => ngrxSerialize(t),
58+
'router redactText (titles, guards, warnings)': (t) => redactText(t),
59+
'router redactRecord (params and data)': (t) => redactRecord({ note: t }),
60+
'errors valueText': (t) => valueText(t),
61+
'errors consoleText': (t) => consoleText(['failed:', t]),
62+
'errors valueText of an Error': (t) => valueText(new Error(t)),
63+
'HTTP call error': (t) => call({ error: t }),
64+
'HTTP response preview (text)': (t) => redactPreview(t),
65+
'HTTP header value': (t) => redactHeaders({ 'x-debug': t }),
66+
'Analog hydration error': (t) => report({ hydrationErrors: [t] }),
67+
'Capacitor plugin argument': (t) => maskedArgs('App', { message: t }, {}),
68+
};
69+
70+
/** Paths that hide value shapes in text, but not `key=value` pairs. */
71+
const SHAPE_ONLY_TEXT_PATHS: Record<string, Path> = {
72+
// Object keys are masked by shape: a key is a name, not a sentence.
73+
'serialize, as an object key': (t) => serialize({ [t]: 1 }),
74+
'NgRx serialize, as an object key': (t) => ngrxSerialize({ [t]: 1 }),
75+
'forms redactMessage (field errors)': (t) => redactMessage(t),
76+
'forms SecretSet (event log)': (t) => new SecretSet().redact(t),
77+
'Analog server log message': (t) => redactServerMessage(t),
78+
'Analog server log body': (t) => redactBody(t),
79+
};
80+
81+
/** Paths that take an object and hide its values by key. */
82+
const OBJECT_PATHS: Record<string, ObjectPath> = {
83+
serialize: (o) => serialize(o),
84+
'NgRx serialize': (o) => ngrxSerialize(o),
85+
'router redactRecord': (o) => redactRecord(o),
86+
'forms serializeFormValue': (o) => serializeFormValue(o),
87+
'HTTP response preview (JSON)': (o) => redactPreview(JSON.stringify(o)),
88+
'HTTP response preview (cut JSON)': (o) => redactPreview(JSON.stringify(o).slice(0, -2)),
89+
'HTTP call preview': (o) => call({ preview: JSON.stringify(o) }),
90+
'JSON text redaction': (o) => redactJsonText(JSON.stringify(o), isRedactedKey),
91+
'Capacitor plugin argument': (o) => maskedArgs('Preferences', o, {}),
92+
'Analog load() preview': (o) =>
93+
report({ load: { preview: JSON.stringify(o), bytes: 10, keys: [] } }),
94+
'Analog server log body': (o) => redactBody(JSON.stringify(o)),
95+
'errors valueText': (o) => valueText(o),
96+
'errors consoleText': (o) => consoleText(['state', o]),
97+
'HTTP headers': (o) =>
98+
redactHeaders(Object.fromEntries(Object.entries(o).map(([k, v]) => [k, String(v)]))),
99+
};
100+
101+
/** Paths that take a URL. */
102+
const URL_PATHS: Record<string, Path> = {
103+
'router redactUrl': (u) => redactUrl(u),
104+
'HTTP call URL': (u) => call({ url: u }),
105+
'HTTP redirect target': (u) => call({ redirectedTo: u }),
106+
'Analog page URL': (u) => report({ url: u }),
107+
'Capacitor HTTP URL': (u) => maskedArgs('CapacitorHttp', { url: u, method: 'GET' }, {}),
108+
'Analog server log message': (u) => redactServerMessage(u),
109+
};
110+
111+
function expectHidden(path: string, input: string, out: unknown, secret: string) {
112+
const text = show(out);
113+
expect(
114+
text,
115+
`${path} leaked ${JSON.stringify(secret)} from ${JSON.stringify(input)}`,
116+
).not.toContain(secret);
117+
expect(text, `${path} hid ${JSON.stringify(input)} without a marker`).toMatch(
118+
/\[redacted\]|%5Bredacted%5D/,
119+
);
120+
}
121+
122+
describe('the leak corpus', () => {
123+
it('has every kind of secret it promises', () => {
124+
expect(corpus.shapes.length).toBeGreaterThanOrEqual(20);
125+
expect(corpus.named.length).toBeGreaterThanOrEqual(10);
126+
for (const entry of corpus.shapes) expect(entry.value).toContain(entry.secret);
127+
});
128+
129+
describe('value shapes hide themselves in any text', () => {
130+
const paths = { ...PAIR_TEXT_PATHS, ...SHAPE_ONLY_TEXT_PATHS };
131+
for (const [path, run] of Object.entries(paths)) {
132+
it(path, () => {
133+
for (const { value, secret } of corpus.shapes) {
134+
for (const input of [value, `request failed (${value}) retry later`]) {
135+
expectHidden(path, input, run(input), secret);
136+
}
137+
}
138+
});
139+
}
140+
});
141+
142+
describe('values under secret names hide in key=value text', () => {
143+
for (const [path, run] of Object.entries(PAIR_TEXT_PATHS)) {
144+
it(path, () => {
145+
for (const { key, secret } of corpus.named) {
146+
for (const input of [
147+
`${key}=${secret}`,
148+
`retry with ${key}=${secret}&user=ada`,
149+
`{"${key}":"${secret}","user":"ada"}`,
150+
`${key}: "${secret}"`,
151+
]) {
152+
expectHidden(path, input, run(input), secret);
153+
}
154+
}
155+
});
156+
}
157+
});
158+
159+
describe('values under secret names hide in objects', () => {
160+
for (const [path, run] of Object.entries(OBJECT_PATHS)) {
161+
it(path, () => {
162+
for (const { key, secret } of corpus.named) {
163+
for (const input of [{ [key]: secret, user: 'ada' }, { form: { [key]: secret } }]) {
164+
if (path === 'HTTP headers' && 'form' in input) continue;
165+
expectHidden(path, JSON.stringify(input), run(input), secret);
166+
}
167+
}
168+
});
169+
}
170+
});
171+
172+
describe('values under secret names hide in URLs', () => {
173+
for (const [path, run] of Object.entries(URL_PATHS)) {
174+
it(path, () => {
175+
for (const { key, secret } of corpus.named) {
176+
for (const input of [
177+
`https://app.example.com/cb?${key}=${secret}&page=2`,
178+
`/login#${key}=${encodeURIComponent(secret)}`,
179+
]) {
180+
expectHidden(path, input, run(input), secret);
181+
}
182+
}
183+
});
184+
}
185+
});
186+
187+
describe('value shapes hide in URLs', () => {
188+
const inUrl = corpus.shapes.filter((entry) => /^(https?:|eyJ)/.test(entry.value));
189+
for (const [path, run] of Object.entries(URL_PATHS)) {
190+
it(path, () => {
191+
for (const { value, secret } of inUrl) {
192+
const input = value.startsWith('eyJ') ? `/verify/${value}?page=2` : value;
193+
expectHidden(path, input, run(input), secret);
194+
}
195+
});
196+
}
197+
});
198+
199+
it('keeps the marker every path writes the same', () => {
200+
expect(REDACTED).toBe('[redacted]');
201+
expect(redactJsonText('{"password":"x"}', isRedactedKey)).toBe(`{"password":"${REDACTED}"}`);
202+
});
203+
});

0 commit comments

Comments
 (0)