Skip to content

Commit 4c361d4

Browse files
authored
feat(overlay): opt-in redaction.entropy masks high-entropy strings (#488)
Adds redaction.entropy (off by default) to mask long high-entropy strings in text, forms and Analog server logs, and passes the resolved options to trusted pages. Refs #371
1 parent 476d925 commit 4c361d4

16 files changed

Lines changed: 881 additions & 134 deletions
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
import { resolvePangularConfig } from '@pangular-inspector/devtools/internal/config';
2+
import { describe, expect, it } from 'vitest';
3+
import { withRedaction } from '../devtools-config';
4+
5+
describe('withRedaction', () => {
6+
const config = resolvePangularConfig();
7+
8+
it('adds the redaction lists the server answers, with entropy only when it is on', () => {
9+
expect(withRedaction(config, { secretNames: ['passport'], unmask: [] }).redaction).toEqual({
10+
secretNames: ['passport'],
11+
unmask: [],
12+
});
13+
expect(
14+
withRedaction(config, { secretNames: [], unmask: [], entropy: { minLength: 32 } }).redaction,
15+
).toEqual({ secretNames: [], unmask: [], entropy: { minLength: 32, threshold: 4.5 } });
16+
expect(
17+
withRedaction(config, { secretNames: [], unmask: [], entropy: 'on' }).redaction,
18+
).not.toHaveProperty('entropy');
19+
});
20+
21+
it('keeps the config for a missing or malformed answer', () => {
22+
expect(withRedaction(config, null)).toBe(config);
23+
expect(withRedaction(config, { secretNames: 'passport', unmask: [] })).toBe(config);
24+
});
25+
});

‎app/src/devtools-config.ts‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ import type { DevframeRpcClient } from 'devframe/client';
22
import {
33
actionBlockedMessage,
44
configFromConnection,
5+
resolveEntropy,
56
type PangularAction,
67
type PangularInspector,
78
type ResolvedPangularConfig,
@@ -34,7 +35,11 @@ export function withRedaction(
3435
const secretNames = names((answer as { secretNames?: unknown }).secretNames);
3536
const unmask = names((answer as { unmask?: unknown }).unmask);
3637
if (!secretNames || !unmask) return config;
37-
return { ...config, redaction: { secretNames, unmask } };
38+
const entropy = resolveEntropy((answer as { entropy?: unknown }).entropy);
39+
return {
40+
...config,
41+
redaction: entropy ? { secretNames, unmask, entropy } : { secretNames, unmask },
42+
};
3843
}
3944

4045
export function actionAllowed(client: DevframeRpcClient | null, action: PangularAction): boolean {

‎apps/docs/src/content/getting-started/configuration.md‎

Lines changed: 42 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,11 @@ interface PangularConfig {
118118
browserTools?: boolean;
119119
};
120120
actions?: boolean | Partial<Record<PangularAction, boolean>>;
121-
redaction?: {secretNames?: string[]; unmask?: string[]};
121+
redaction?: {
122+
secretNames?: string[];
123+
unmask?: string[];
124+
entropy?: boolean | {minLength?: number; threshold?: number};
125+
};
122126
limits?: {
123127
refreshMs?: number;
124128
navigations?: number;
@@ -174,12 +178,30 @@ The server refuses a blocked action, and with `actions.components` the page refu
174178

175179
### Redaction
176180

177-
| Option | Default | What it does |
178-
| ----------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
179-
| `redaction.secretNames` | `[]` | Extra field names to treat as secret, on top of the built-in list. They match by words, like the built-in list, so `passport` also covers `passportNumber`. |
180-
| `redaction.unmask` | `[]` | Field names to show even when they look secret. They join the `unmask` list of `window.__PANGULAR_FORMS__`. |
181+
| Option | Default | What it does |
182+
| ----------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
183+
| `redaction.secretNames` | `[]` | Extra field names to treat as secret, on top of the built-in list. They match by words, like the built-in list, so `passport` also covers `passportNumber`. |
184+
| `redaction.unmask` | `[]` | Field names to show even when they look secret. They join the `unmask` list of `window.__PANGULAR_FORMS__`. |
185+
| `redaction.entropy` | `false` | Also mask high-entropy strings that no name or shape rule catches, such as an opaque token in a harmless-looking field. Hex values and ULIDs stay readable. `true` turns it on with the defaults; an object turns it on and sets the options below. |
186+
187+
| `redaction.entropy` field | Default | What it sets |
188+
| ------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
189+
| `minLength` | `24` | The shortest run that can be masked, a whole number from 16 to 512. Raise it if ids in your app are masked that shouldn't be. |
190+
| `threshold` | `4.5` | The most bits per character a run needs, from 2 to 6. Shorter runs need less (see [High-entropy strings](../security.md#high-entropy-strings)). |
191+
192+
A field that is out of range takes its default, and the server reports the problem when it starts. To mask only longer runs:
193+
194+
```ts
195+
// vite.config.ts
196+
import pangular from '@pangular-inspector/devtools/vite';
197+
import {defineConfig} from 'vite';
198+
199+
export default defineConfig({
200+
plugins: [pangular({redaction: {entropy: {minLength: 32}}})],
201+
});
202+
```
181203

182-
Forms, the router, component inputs, signals, NgRx, pipes, Analog previews, SSR & HTTP URLs and Capacitor plugin calls use the extra secret names. Each list keeps up to 100 names. The server sends both lists to a page only after the page is trusted, so they are not in `__connection.json`. See [Access and redaction](../security.md#what-is-redacted) for what is redacted and what unmasking allows.
204+
Forms, the router, component inputs, signals, NgRx, pipes, Analog previews, SSR & HTTP URLs and Capacitor plugin calls use the extra secret names. Each list keeps up to 100 names. `redaction.entropy` applies to the same text, on the page and again on the devtools server; [High-entropy strings](../security.md#high-entropy-strings) says what it masks. The server sends the `redaction` options to a page only after the page is trusted, so they are not in `__connection.json`. See [Access and redaction](../security.md#what-is-redacted) for what is redacted and what unmasking allows.
183205

184206
### Limits
185207

@@ -330,19 +352,20 @@ A value that the server ignores falls back to its default, which is usually the
330352

331353
The [Dashboard](../inspectors/dashboard.md#configuration-block) has a **Configuration** block. It lists the options that differ from the defaults, or says **Defaults** when nothing is changed. The panel reads the `redaction` rows from the server once it is trusted, since `__connection.json` leaves them out. A static report has no server to ask, so it shows no **Extra secret names** or **Unmasked names** rows.
332354

333-
| Row | Lists |
334-
| ------------------------------- | ------------------------------------------------- |
335-
| **Inspectors off** | Inspectors set to `false`. |
336-
| **Agent** | **Read-only** when `agent.readOnly` is `true`. |
337-
| **Hidden from the agent** | Inspectors whose agent tools are hidden. |
338-
| **Blocked actions** | Actions set to `false`. |
339-
| **Extra secret names** | `redaction.secretNames`. |
340-
| **Unmasked names** | `redaction.unmask`. |
341-
| **Limits** | Limits that differ from the defaults. |
342-
| **Popup shortcut** | `popup.hotkey`, or **Off** when it is `false`. |
343-
| **Plugins off** | Plugins set to `false` in `plugins`. |
344-
| **Plugin tools or actions off** | Plugins with `tools` or `actions` set to `false`. |
345-
| **Lint rules off** | Rules set to `false` in `lint.rules`. |
355+
| Row | Lists |
356+
| ------------------------------- | -------------------------------------------------------------------------------------- |
357+
| **Inspectors off** | Inspectors set to `false`. |
358+
| **Agent** | **Read-only** when `agent.readOnly` is `true`. |
359+
| **Hidden from the agent** | Inspectors whose agent tools are hidden. |
360+
| **Blocked actions** | Actions set to `false`. |
361+
| **Extra secret names** | `redaction.secretNames`. |
362+
| **Unmasked names** | `redaction.unmask`. |
363+
| **Entropy masking** | **On** when `redaction.entropy` is on, with the options that differ from the defaults. |
364+
| **Limits** | Limits that differ from the defaults. |
365+
| **Popup shortcut** | `popup.hotkey`, or **Off** when it is `false`. |
366+
| **Plugins off** | Plugins set to `false` in `plugins`. |
367+
| **Plugin tools or actions off** | Plugins with `tools` or `actions` set to `false`. |
368+
| **Lint rules off** | Rules set to `false` in `lint.rules`. |
346369

347370
## Where to next
348371

‎apps/docs/src/content/security.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -299,6 +299,26 @@ An email address in a URL is replaced with `[email]`, in a path segment, a query
299299

300300
An `@` that isn't part of an address stays, as in `/@angular/core`, `[email protected]` or `/users/@alice`. The user in a URL's authority, such as `git` in `ssh://[email protected]`, stays visible like the user before a URL password. Inside an encoded URL in a query value (`?next=https%3A%2F%2Fbob%40example.com`) there is no authority to tell apart, so the address is masked, host included. The router lint (`lint-routes` and the [Lint](./inspectors/router.md#lint) view) still reports the `email-in-url` rule for a navigation whose address was masked.
301301

302+
### High-entropy strings
303+
304+
Set `redaction.entropy` to `true` to also mask strings that look random, such as an opaque session token in a field with a harmless name or a cloud secret with no known prefix. It is off by default. An object such as `{minLength: 32, threshold: 4}` turns it on with other limits. See [Redaction options](./getting-started/configuration.md#redaction).
305+
306+
A run of letters, digits, `_`, `-`, `+` and `/` (and any trailing `=`) is replaced with `[redacted]` when all of these hold:
307+
308+
| Check | Why |
309+
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------ |
310+
| It has `minLength` characters or more (24) | Short ids, words and numbers stay readable. |
311+
| It mixes lowercase, uppercase and digits | Plain words, numbers and lowercase slugs such as `css-1a2b3c` stay readable. |
312+
| It is not hex and not a ULID | Hashes, commit SHAs, UUIDs, Mongo ids and ULIDs stay readable. |
313+
| It doesn't read as words | camelCase and snake_case identifiers, such as `getUserProfileById2024`, stay readable. |
314+
| Its Shannon entropy is high enough | `log2(length) - 1` bits per character, at most `threshold` (4.5): 3.6 bits at 24 characters, 4 at 32, 4.5 from 45. |
315+
316+
About 99 in 100 random 24-character tokens pass these checks, and more of longer ones. A run with `/` is judged whole, as base64 is, unless one of its segments is a plain word or number, as in a path. Then each segment is judged on its own, so `/api/v2/Users/<token>/edit` keeps its path readable. Long base64 data, such as an inline image, is masked too.
317+
318+
The check runs last, after the name and shape rules, so a Stripe key still shows its `sk_live_` prefix. It covers every text the other rules cover: values, keys, messages, URLs, previews and headers in serialized values, NgRx state, forms, the router, SSR & HTTP calls, Analog reports and server logs, Capacitor calls and errors, on the page and again on the devtools server.
319+
320+
The check reads each text once, so its cost grows in step with the text, and a long text of 200 KB takes milliseconds to check.
321+
302322
### Router
303323

304324
These are replaced with `[redacted]` in URLs, params, data and messages:

extension/ui/assets/browser-agent-rpc-KH4HHvHf-D-916cEb.js renamed to extension/ui/assets/browser-agent-rpc-KH4HHvHf-BeC-w14M.js

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 105 additions & 105 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎extension/ui/index.html‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎packages/devtools/CHANGELOG.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -135,7 +135,8 @@ What remains: `/config` exports `DEFAULT_POPUP_HOTKEY`, `PANGULAR_ACTIONS`, `PAN
135135
- A **Terminals** dock that runs the project's build and test commands (opt in with `terminals: true`).
136136
- Hub messages and toasts, a dock group with badges, and an **All tools** dock.
137137
- New entry points: `/overlay-manual` (starts nothing on import), `/plugin`, `/capacitor-hook`, `/store-hook`, `/startup-hook`, `/ssr` and `/cd-data`.
138-
- New config options: `agent.browserTools`, `limits.capacitorCalls`, `limits.idleAfterMs`, `popup` and `plugins`.
138+
- New config options: `agent.browserTools`, `limits.capacitorCalls`, `limits.idleAfterMs`, `popup`, `plugins` and `redaction.entropy`.
139+
- `redaction.entropy`, off by default, masks long high-entropy strings that no name or shape rule catches, such as an opaque token in a harmless field. `true` masks runs of 24 characters or more; `{minLength, threshold}` sets other limits. Hex values, UUIDs, ULIDs and camelCase identifiers stay readable, and known shapes keep their prefix. It applies on every redaction path, on the page and on the devtools server, and shows as **Entropy masking** on the Dashboard.
139140
- After pairing, focus moves to the active tab with a **Connected** note.
140141

141142
#### Inspectors

0 commit comments

Comments
 (0)