Skip to content

Commit 37a69f0

Browse files
AgentEndernx-cloud[bot]
AgentEnder
and
nx-cloud[bot]
authored
feat(core): eagerly shutdown plugins that don't provide later hooks (#34253)
# Plugin Isolation Architecture ## 1. Plugin Loading Flow ### 1a. Entry Point - Isolation Decision ```mermaid flowchart TD Start([getPlugins called]) --> CheckIsolation{Isolation<br/>enabled?} CheckIsolation -->|Yes| LoadIsolated[loadIsolatedNxPlugin] CheckIsolation -->|No| LoadInProcess[loadNxPluginInProcess] LoadIsolated --> IsolatedPath([See: Isolated Loading]) LoadInProcess --> InProcessPath([See: In-Process Loading]) ``` ### 1b. Isolated Plugin Loading ```mermaid flowchart TD subgraph Main["Main Process"] Start([loadIsolatedNxPlugin]) --> CheckCache{In cache?} CheckCache -->|Yes| ReturnCached([Return cached promise]) CheckCache -->|No| StaticLoad[IsolatedPlugin.load] StaticLoad --> Resolve[resolveNxPlugin<br/>find plugin path] Resolve --> SpawnWorker[spawn child process] end SpawnWorker -.->|"start process"| WorkerStart subgraph Worker["Worker Process (plugin-worker.ts)"] WorkerStart([process starts]) --> CreateServer[create Unix socket server] CreateServer --> Listen[listen for connections] Listen --> WaitForConnect[wait for main process] WaitForConnect --> HandleLoad[receive 'load' message] subgraph InProcess["In-Process Loading (same as 1c)"] HandleLoad --> RequirePlugin[require plugin module] RequirePlugin --> NormalizePlugin[normalizeNxPlugin] end NormalizePlugin --> SendLoadResult[send 'loadResult'<br/>with hook capabilities] SendLoadResult --> WaitForMessages[wait for hook messages<br/>or socket close] WaitForMessages --> HandleHook{message<br/>received?} HandleHook -->|hook message| ExecuteHook[call plugin.hook] ExecuteHook --> SendResult[send result] SendResult --> WaitForMessages HandleHook -->|socket closed| Cleanup[cleanup & exit] end subgraph Main2["Main Process (continued)"] ConnectSocket[connect via<br/>Unix socket] --> SendLoad[send 'load' message] SendLoad --> WaitLoad[wait for 'loadResult'] WaitLoad --> SetupHooks[setupHooks<br/>create lifecycle manager] SetupHooks --> CheckGraphHooks{Has graph<br/>phase hooks?} CheckGraphHooks -->|No| EarlyShutdown[socket.end<br/>shutdown worker] CheckGraphHooks -->|Yes| KeepAlive[keep worker alive] EarlyShutdown --> Done([Plugin ready]) KeepAlive --> Done end SpawnWorker --> ConnectSocket SendLoadResult -.->|"loadResult"| WaitLoad EarlyShutdown -.->|"socket close"| Cleanup ``` ### 1c. In-Process Plugin Loading ```mermaid flowchart TD Start([loadNxPluginInProcess]) --> Resolve[resolveNxPlugin] Resolve --> Require[require plugin module] Require --> Normalize[normalizeNxPlugin<br/>wrap hooks] Normalize --> Done([Plugin ready]) ``` ## 2. Hook Execution Flow ### 2a. Isolated Hook Execution ```mermaid flowchart TD Start([hook called<br/>e.g. createNodes]) --> EnsureAlive{_alive?} EnsureAlive -->|No| Restart[spawnAndConnect<br/>restart worker] Restart --> SetAlive[_alive = true] SetAlive --> EnsureAlive EnsureAlive -->|Yes| EnterHook[lifecycle.enterHook<br/>increment session count] EnterHook --> SendRequest[sendRequest<br/>over socket] SendRequest --> WaitResponse[wait for response<br/>with timeout] WaitResponse --> CheckSuccess{success?} CheckSuccess -->|No| ExitHookError[lifecycle.exitHook] ExitHookError --> ThrowError[throw error] CheckSuccess -->|Yes| ExitHook[lifecycle.exitHook] ExitHook --> CheckShutdown{should<br/>shutdown?} CheckShutdown -->|Yes| Shutdown[shutdown worker] CheckShutdown -->|No| Return([return result]) Shutdown --> Return ``` ### 2b. Shutdown Decision Logic ```mermaid flowchart TD Start([exitHook called]) --> IsLastHook{Last hook<br/>in phase?} IsLastHook -->|No| NoShutdown1([return false]) IsLastHook -->|Yes| CheckSessions{sessionCount<br/>== 0?} CheckSessions -->|No| NoShutdown2([return false<br/>other callers active]) CheckSessions -->|Yes| CheckLaterPhases{Has later<br/>active phases?} CheckLaterPhases -->|Yes| NoShutdown3([return false<br/>needed later]) CheckLaterPhases -->|No| YesShutdown([return true<br/>safe to shutdown]) ``` ## 3. Developer Workflow: Adding/Modifying Plugin Hooks ### Step 1: Design Public API ```mermaid flowchart TD A1[public-api.ts] --> A2[Define context type<br/>e.g. MyHookContext] A2 --> A3[Export new types] A3 --> A4[loaded-nx-plugin.ts] A4 --> A5[Add hook to<br/>LoadedNxPlugin interface] ``` ### Step 2: Define Message Types ```mermaid flowchart TD B1[messaging.ts] --> B2[Add entry to PluginMessageDefs] B2 --> B3[Define payload and result types] B3 --> B4[Add to MESSAGE_TYPES array] B4 --> B5[Add to RESULT_TYPES array] ``` The messaging system uses a unified `DefineMessages` pattern. To add a new message: ```typescript // In PluginMessageDefs, add a new entry: type PluginMessageDefs = DefineMessages<{ // ... existing messages ... myHook: { payload: { context: MyHookContext; }; result: | { success: true; data: MyResultData } | { success: false; error: Error }; }; }>; ``` The individual message/result types (`PluginWorkerMyHookMessage`, `PluginWorkerMyHookResult`) are automatically derived. Export them if needed for external use: ```typescript export type PluginWorkerMyHookMessage = MessageOf<PluginMessageDefs, 'myHook'>; export type PluginWorkerMyHookResult = ResultOf<PluginMessageDefs, 'myHook'>; ``` ### Step 3: Handle in Worker Process ```mermaid flowchart TD C1[plugin-worker.ts] --> C2[Add handler in<br/>consumeMessage] C2 --> C3["Call plugin.myHook()"] C3 --> C4[Return result payload] ``` Handlers return just the result payload - the infrastructure wraps it automatically: ```typescript // In consumeMessage handlers: myHook: async ({ context }) => { try { const data = await plugin.myHook(context); return { success: true as const, data }; } catch (e) { return { success: false as const, error: createSerializableError(e) }; } }, ``` ### Step 4: Update Load Result ```mermaid flowchart TD D1[messaging.ts] --> D2[Add hasMyHook to<br/>load.result in PluginMessageDefs] D2 --> D3[plugin-worker.ts] D3 --> D4[Populate hasMyHook<br/>in load handler] ``` ### Step 5: Wire Up IsolatedPlugin ```mermaid flowchart TD E1[isolated-plugin.ts] --> E2[Add hook property<br/>to class] E2 --> E3[Update LoadResultPayload<br/>type export] E3 --> E4[Add to registeredHooks<br/>array in setupHooks] E4 --> E5[Add wrapped hook<br/>implementation] E5 --> E6["wrap('myHook', async (ctx) => {<br/> sendRequest('myHook', { context: ctx })<br/>})"] ``` ### Step 6: Update Lifecycle Phases (if needed) ```mermaid flowchart TD F1{New phase<br/>needed?} -->|Yes| F2[plugin-lifecycle-manager.ts] F2 --> F3[Add phase to<br/>HOOKS_BY_PHASE] F1 -->|No| F4[Add hook to existing<br/>phase array in HOOKS_BY_PHASE] ``` ### Step 7: Add Tests ```mermaid flowchart TD G1[isolated-plugin.spec.ts] --> G2[Test hook registration] G2 --> G3[Test hook execution] G3 --> G4[Test restart behavior] G4 --> G5[plugin-lifecycle-manager.spec.ts] G5 --> G6[Test phase transitions<br/>with new hook] G6 --> G7[Test shutdown decisions] ``` ## File Reference | File | Purpose | | ----------------------------- | ------------------------------------------------------------- | | `../public-api.ts` | Public types exported to plugin authors | | `../loaded-nx-plugin.ts` | Interface definition for loaded plugins | | `messaging.ts` | Message type definitions for worker communication | | `plugin-worker.ts` | Worker process - receives messages, calls plugin functions | | `isolated-plugin.ts` | Main class - spawns worker, sends messages, manages lifecycle | | `plugin-lifecycle-manager.ts` | Tracks phases, decides when to shutdown | | `load-isolated-plugin.ts` | Caching layer for isolated plugins | | `../get-plugins.ts` | Entry point - decides isolation mode | ## Lifecycle Phases ``` LOADED → [graph] → [pre-task] → {tasks run} → [post-task] │ │ │ │ └── preTasksExecution ──────┤ │ │ ├── createNodes │ ├── createDependencies │ └── createMetadata │ │ postTasksExecution ``` **Shutdown rules:** - Plugin shuts down after its last active phase completes - If only `postTasksExecution`: shutdown immediately after load, restart when needed - Concurrent callers tracked via session count (ref counting) --------- Co-authored-by: nx-cloud[bot] <71083854+nx-cloud[bot]@users.noreply.github.com> Co-authored-by: AgentEnder <[email protected]>
1 parent 359c7fb commit 37a69f0

19 files changed

Lines changed: 2113 additions & 968 deletions

‎e2e/release/src/independent-projects.test.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -238,7 +238,7 @@ describe('nx release - independent projects', () => {
238238
`release version 999.9.9-version-git-operations-test.2 -p ${pkg1} --git-commit --git-tag --verbose` // add verbose so we get richer output
239239
);
240240
const filteredOutput = versionWithGitActionsCLIOutput.replace(
241-
/\[plugin-(pool|worker)\].*\n/g,
241+
/\[(isolated-plugin|plugin-worker)\].*\n/g,
242242
''
243243
);
244244
expect(filteredOutput).toMatchInlineSnapshot(`
@@ -325,7 +325,7 @@ describe('nx release - independent projects', () => {
325325
`release version 999.9.9-version-git-operations-test.3 --verbose --gitTag` // add verbose so we get richer output
326326
);
327327
const filteredConfigOutput = versionWithGitActionsConfigOutput.replace(
328-
/\[plugin-(pool|worker)\].*\n/g,
328+
/\[(isolated-plugin|plugin-worker)\].*\n/g,
329329
''
330330
);
331331
expect(filteredConfigOutput).toMatchInlineSnapshot(`
@@ -525,7 +525,7 @@ describe('nx release - independent projects', () => {
525525
`release changelog 999.9.9-changelog-git-operations-test.1 -p ${pkg1} --verbose`
526526
);
527527
const filteredChangelogOutput = versionWithGitActionsCLIOutput.replace(
528-
/\[plugin-(pool|worker)\].*\n/g,
528+
/\[(isolated-plugin|plugin-worker)\].*\n/g,
529529
''
530530
);
531531
expect(filteredChangelogOutput).toMatchInlineSnapshot(`

‎e2e/release/src/independent-projects.workspaces.test.ts‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,20 +2,18 @@ import { joinPathFragments, NxJsonConfiguration } from '@nx/devkit';
22
import {
33
cleanupProject,
44
exists,
5-
getSelectedPackageManager,
65
getPackageManagerCommand,
6+
getSelectedPackageManager,
77
newProject,
88
readFile,
99
runCLI,
1010
runCommand,
11-
runCommandAsync,
1211
tmpProjPath,
1312
uniq,
1413
updateJson,
15-
removeFile,
1614
} from '@nx/e2e-utils';
1715
import { execSync } from 'child_process';
18-
import { setupWorkspaces, prepareAndInstallDependencies } from './utils';
16+
import { prepareAndInstallDependencies, setupWorkspaces } from './utils';
1917

2018
expect.addSnapshotSerializer({
2119
serialize(str: string) {
@@ -55,7 +53,7 @@ expect.addSnapshotSerializer({
5553
.replaceAll(getSelectedPackageManager(), '{package-manager}')
5654
.replaceAll(e2eRegistryUrl, '{registryUrl}')
5755
// Filter out plugin worker verbose logs
58-
.replaceAll(/\[plugin-(pool|worker)\].*\n/g, '')
56+
.replaceAll(/\[(isolated-plugin|plugin-worker)\].*\n/g, '')
5957
// We trim each line to reduce the chances of snapshot flakiness
6058
.split('\n')
6159
.map((r) => r.trim())

‎e2e/release/src/preserve-local-dependency-protocols.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ expect.addSnapshotSerializer({
4848
'Integrity: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
4949
)
5050
// Filter out plugin worker verbose logs
51-
.replaceAll(/\[plugin-(pool|worker)\].*\n/g, '')
51+
.replaceAll(/\[(isolated-plugin|plugin-worker)\].*\n/g, '')
5252

5353
.split('\n')
5454
.map((r) => r.trim())

‎e2e/release/src/version-plans-check.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ expect.addSnapshotSerializer({
3333
// Normalize the version title date.
3434
.replaceAll(/\(\d{4}-\d{2}-\d{2}\)/g, '(YYYY-MM-DD)')
3535
// Filter out plugin worker verbose logs
36-
.replaceAll(/\[plugin-(pool|worker)\].*\n/g, '')
36+
.replaceAll(/\[(isolated-plugin|plugin-worker)\].*\n/g, '')
3737
// We trim each line to reduce the chances of snapshot flakiness
3838
.split('\n')
3939
.map((r) => r.trim())

‎e2e/release/src/version-plans-only-touched.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ expect.addSnapshotSerializer({
3030
// Normalize the version title date.
3131
.replaceAll(/\(\d{4}-\d{2}-\d{2}\)/g, '(YYYY-MM-DD)')
3232
// Filter out plugin worker verbose logs
33-
.replaceAll(/\[plugin-(pool|worker)\].*\n/g, '')
33+
.replaceAll(/\[(isolated-plugin|plugin-worker)\].*\n/g, '')
3434
// We trim each line to reduce the chances of snapshot flakiness
3535
.split('\n')
3636
.map((r) => r.trim())

‎packages/nx/src/project-graph/plugins/get-plugins.ts‎

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,17 +3,16 @@ import { join } from 'node:path';
33
import { shouldMergeAngularProjects } from '../../adapter/angular-json';
44
import { PluginConfiguration, readNxJson } from '../../config/nx-json';
55
import { hashObject } from '../../hasher/file-hasher';
6-
import { IS_WASM } from '../../native';
76
import { workspaceRoot } from '../../utils/workspace-root';
8-
import { loadNxPluginInIsolation } from './isolation';
97
import { loadNxPlugin } from './in-process-loader';
8+
import { loadIsolatedNxPlugin } from './isolation';
109

10+
import { isIsolationEnabled } from './isolation/enabled';
1111
import type { LoadedNxPlugin } from './loaded-nx-plugin';
1212
import {
1313
cleanupPluginTSTranspiler,
1414
pluginTranspilerIsRegistered,
1515
} from './transpiler';
16-
import { isIsolationEnabled } from './isolation/enabled';
1716

1817
/**
1918
* Stuff for specified NX Plugins.
@@ -29,8 +28,8 @@ const loadingMethod = (
2928
index?: number
3029
) =>
3130
isIsolationEnabled()
32-
? loadNxPluginInIsolation(plugin, root, index)
33-
: loadNxPlugin(plugin, root);
31+
? loadIsolatedNxPlugin(plugin, root, index)
32+
: loadNxPlugin(plugin, root, index);
3433

3534
export async function getPlugins(
3635
root = workspaceRoot
@@ -234,7 +233,6 @@ async function loadSpecifiedNxPlugins(
234233

235234
cleanupFunctions.push(cleanup);
236235
const res = await loadedPluginPromise;
237-
res.index = index;
238236
performance.mark(`Load Nx Plugin: ${pluginPath} - end`);
239237
performance.measure(
240238
`Load Nx Plugin: ${pluginPath}`,

‎packages/nx/src/project-graph/plugins/in-process-loader.ts‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -54,17 +54,22 @@ export function readPluginPackageJson(
5454
}
5555
}
5656

57-
export function loadNxPlugin(plugin: PluginConfiguration, root: string) {
57+
export function loadNxPlugin(
58+
plugin: PluginConfiguration,
59+
root: string,
60+
index?: number
61+
) {
5862
return [
59-
loadNxPluginAsync(plugin, getNxRequirePaths(root), root),
63+
loadNxPluginAsync(plugin, getNxRequirePaths(root), root, index),
6064
() => {},
6165
] as const;
6266
}
6367

6468
export async function loadNxPluginAsync(
6569
pluginConfiguration: PluginConfiguration,
6670
paths: string[],
67-
root: string
71+
root: string,
72+
index?: number
6873
): Promise<LoadedNxPlugin> {
6974
const moduleName =
7075
typeof pluginConfiguration === 'string'
@@ -80,7 +85,12 @@ export async function loadNxPluginAsync(
8085
const { loadResolvedNxPluginAsync } = await import(
8186
'./load-resolved-plugin'
8287
);
83-
return loadResolvedNxPluginAsync(pluginConfiguration, pluginPath, name);
88+
return loadResolvedNxPluginAsync(
89+
pluginConfiguration,
90+
pluginPath,
91+
name,
92+
index
93+
);
8494
} catch (e) {
8595
throw new LoadPluginError(moduleName, e);
8696
}

‎packages/nx/src/project-graph/plugins/isolation/index.ts‎

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1 @@
1-
import { workspaceRoot } from '../../../utils/workspace-root';
2-
import type { PluginConfiguration } from '../../../config/nx-json';
3-
import type { LoadedNxPlugin } from '../loaded-nx-plugin';
4-
import { loadRemoteNxPlugin } from './plugin-pool';
5-
6-
export async function loadNxPluginInIsolation(
7-
plugin: PluginConfiguration,
8-
root = workspaceRoot,
9-
index?: number
10-
): Promise<readonly [Promise<LoadedNxPlugin>, () => void]> {
11-
return loadRemoteNxPlugin(plugin, root, index);
12-
}
1+
export { loadIsolatedNxPlugin } from './load-isolated-plugin';

0 commit comments

Comments
 (0)