Persist the spec of the computing_unit entity and its relationship with workflow_executions into DB - #3400

…with `workflow_executions` into DB (apache#3400)

This PR persists the computing unit information to the Database and
uniforms the underlying implementation of single mode(with local
computing master as the computing unit) and cluster mode(with kubernetes
pod as the computing unit).

## For Developers

**Please do the following steps to incorporate with new changes**:
1. Apply `core/scripts/sql/updates/07.sql` to your local postgres
instance
2. Always starts `ComputingUnitManagingService` when you launch the
Texera's backend micro-services.

## Database Schema Changes

### Modification of `computing_unit` Table
The following fields have been added:
- `type`: Type of computing unit (e.g., "local" or "kubernetes").
- `resource`: A json string contains:
    - cpu_limit: CPU resource limit allocated to the computing unit.
- memory_limit: Memory resource limit allocated to the computing unit.
    - gpu_limit: GPU resource limit allocated to the computing unit.
    - jvm_memory_size: JVM memory size allocated to the computing unit.
- node_addresses: List of node addresses associated with the computing
unit.


### Modification of `workflow_execution` Table
A new column `cuid` has been added to the existing workflow_execution
table to establish a relationship between workflow executions and
computing units. This column serves as a foreign key referencing the
`cuid` in the `computing_unit` table.

## Configuration File Changes
### `computing-unit.conf` has been added
A variable `computing-unit.local.enabled` is added. The default value is
`true`. In cluster mode, this should be set to `false`.

### `kubernetes.conf` has been changed
A variable `kubernetes.enabled` is added. The default value is `false`.
In cluster mode, this should be set to `true`.


## User Experience

### 1. Cluster Mode

- Run & interact with the workflow on different computing units at the
same time
![2025-05-12 21 47
14](https://github.com/user-attachments/assets/044d13b0-8bb1-45b6-8e84-f4fdfaf78783)

- Check which computing unit is used in the execution via the execution
history table
![2025-05-12 21 49
21](https://github.com/user-attachments/assets/96c94b52-8fc2-4343-8d77-60775ab98794)

- **If a workflow is running in some computing units, the workflow
CANNOT be modified until there are no RUNNING executions**
![2025-05-12 21 51
56](https://github.com/user-attachments/assets/83daaabc-4444-46fe-9bcc-9da85ef7e208)


### 2. Single Node Mode (Local Development)

Single node Mode needs a little extra step of connecting to a computing
unit: in the pop-up window, users can input an address that points to
the computing unit master service (the default value is
http://localhost:8085)
![2025-05-12 21 41
15](https://github.com/user-attachments/assets/691048bc-f8c0-4d55-a4de-53b23b7c3bc3)

Users can also disconnect from the local computing unit by clicking on
the delete button in the drop-down item:
![2025-05-12 21 41
32](https://github.com/user-attachments/assets/e4952155-9729-47b9-b8e3-1daaf4f35e72)

## Related Changes
### 1. Helm Chart
- `KUBERNETES_COMPUTING_UNIT_ENABLED` and `COMPUTING_UNIT_LOCAL_ENABLED`
have been added
- request to the endpoint `/api/executions/{wid}/stats/{eid}` will be
routed to `envoy`

### 2. Execution Lifecycle
- The latest execution of a workflow is now identified by BOTH `wid` and
`cuid`, i.e. the latest execution of a workflow on a computing unit.
- Every time a new execution starts, the previous execution **on the
same computing unit** will be cleared. The latest executions on other
computing units will NOT be cleared.

### 3. Execution & Result Lifecycle Binding
- Currently, the lifecycle of the execution results and computing units
are bundled.
- As a consequence, querying the result, including port result tables,
stats, and console message, need to carry the `cuid`

…with `workflow_executions` into DB (apache#3400)

This PR persists the computing unit information to the Database and
uniforms the underlying implementation of single mode(with local
computing master as the computing unit) and cluster mode(with kubernetes
pod as the computing unit).

## For Developers

**Please do the following steps to incorporate with new changes**:
1. Apply `core/scripts/sql/updates/07.sql` to your local postgres
instance
2. Always starts `ComputingUnitManagingService` when you launch the
Texera's backend micro-services.

## Database Schema Changes

### Modification of `computing_unit` Table
The following fields have been added:
- `type`: Type of computing unit (e.g., "local" or "kubernetes").
- `resource`: A json string contains:
    - cpu_limit: CPU resource limit allocated to the computing unit.
- memory_limit: Memory resource limit allocated to the computing unit.
    - gpu_limit: GPU resource limit allocated to the computing unit.
    - jvm_memory_size: JVM memory size allocated to the computing unit.
- node_addresses: List of node addresses associated with the computing
unit.


### Modification of `workflow_execution` Table
A new column `cuid` has been added to the existing workflow_execution
table to establish a relationship between workflow executions and
computing units. This column serves as a foreign key referencing the
`cuid` in the `computing_unit` table.

## Configuration File Changes
### `computing-unit.conf` has been added
A variable `computing-unit.local.enabled` is added. The default value is
`true`. In cluster mode, this should be set to `false`.

### `kubernetes.conf` has been changed
A variable `kubernetes.enabled` is added. The default value is `false`.
In cluster mode, this should be set to `true`.


## User Experience

### 1. Cluster Mode

- Run & interact with the workflow on different computing units at the
same time
![2025-05-12 21 47
14](https://github.com/user-attachments/assets/044d13b0-8bb1-45b6-8e84-f4fdfaf78783)

- Check which computing unit is used in the execution via the execution
history table
![2025-05-12 21 49
21](https://github.com/user-attachments/assets/96c94b52-8fc2-4343-8d77-60775ab98794)

- **If a workflow is running in some computing units, the workflow
CANNOT be modified until there are no RUNNING executions**
![2025-05-12 21 51
56](https://github.com/user-attachments/assets/83daaabc-4444-46fe-9bcc-9da85ef7e208)


### 2. Single Node Mode (Local Development)

Single node Mode needs a little extra step of connecting to a computing
unit: in the pop-up window, users can input an address that points to
the computing unit master service (the default value is
http://localhost:8085)
![2025-05-12 21 41
15](https://github.com/user-attachments/assets/691048bc-f8c0-4d55-a4de-53b23b7c3bc3)

Users can also disconnect from the local computing unit by clicking on
the delete button in the drop-down item:
![2025-05-12 21 41
32](https://github.com/user-attachments/assets/e4952155-9729-47b9-b8e3-1daaf4f35e72)

## Related Changes
### 1. Helm Chart
- `KUBERNETES_COMPUTING_UNIT_ENABLED` and `COMPUTING_UNIT_LOCAL_ENABLED`
have been added
- request to the endpoint `/api/executions/{wid}/stats/{eid}` will be
routed to `envoy`

### 2. Execution Lifecycle
- The latest execution of a workflow is now identified by BOTH `wid` and
`cuid`, i.e. the latest execution of a workflow on a computing unit.
- Every time a new execution starts, the previous execution **on the
same computing unit** will be cleared. The latest executions on other
computing units will NOT be cleared.

### 3. Execution & Result Lifecycle Binding
- Currently, the lifecycle of the execution results and computing units
are bundled.
- As a consequence, querying the result, including port result tables,
stats, and console message, need to carry the `cuid`

…e#8827)

### What changes were proposed in this PR?

Removes two leftovers from `ComputingUnitCreateModalComponent`. **−2
lines.** There is no behaviour change.

- `<ng-template #addComputeUnitModalTitle>`. The modal's title comes
from `[nzTitle]="getCreateModalTitle()"`. Nothing else reads the
template: there is no `@ViewChild`, no outlet, and neither of the two
hosts that open the modal passes it. `nz-modal` only picks up its
title/content/footer directives, never a plain `#ref` template.
- The `selectedShmSize` field. The create payload builds `shmSize` from
`shmSizeValue` / `shmSizeUnit`, which stay.

### History

| | |
| --- | --- |
| **Introduced by** | apache#3315 (2025-03-13), "Add computing unit manager
frontend": the title template. apache#3451 (2025-05-28), "Enable adjusting
shared memory for the computing unit": `selectedShmSize` |
| **Usage removed by** | apache#3400 (2025-05-26) switched the title to
`getCreateModalTitle()`. apache#4331 (2026-04-09), "add full computing unit
tab", dropped the field's last write and read. apache#6082 later moved both,
still unread, into this shared component |

The title template has been unused for sixteen months, and the field for
six.

### Any related issues, documentation, discussions?

Closes apache#8826

### How was this PR tested?

No new tests; no spec referenced either item.

From `frontend/`:

- `npx ng test --watch=false
--include='**/computing-unit-create-modal.component.spec.ts'
--include='**/user-computing-unit.component.spec.ts'
--include='**/computing-unit-selection.component.spec.ts'`: 3 files, 248
tests, all pass. The other two specs cover the modal's two hosts.
- `yarn format:ci`: clean.
- `npx ng build`: success.

To re-check:

```
git grep -n "addComputeUnitModalTitle\|selectedShmSize" -- frontend   # no hits
```

### Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5.5)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

)

### What changes were proposed in this PR?

**Root cause.** When a workflow opens,
`ComputingUnitSelectionComponent.selectFromLastExecution` selects the
unit the workflow last ran on without checking that the unit is still
listed. `/api/executions/{wid}/latest` also returns terminated units, so
`ComputingUnitStatusService.selectComputingUnit` waits forever for a
`cuid` that never appears, and the workflow opens with no unit selected
even while another unit is Running. The "first Running unit" fallback
only ran when the lookup failed, and it read the unit list
synchronously, so it could select nothing if the list had not loaded
yet.

```
Before:  open workflow -> last execution's unit was terminated -> wait for it forever  -> no unit selected
After:   open workflow -> last execution's unit is not listed   -> first Running unit  -> ready to run
```

Both screenshots: the workflow last ran on "Analytics Unit", which was
then terminated, and the workflow was reloaded. "My Computing Unit" is
Running.

| Before | After |
| --- | --- |
| <img width="1512" height="805"
alt="restore-unit-2-reload-after-terminate-nothing-selected"
src="https://github.com/user-attachments/assets/8909822d-957b-465e-9adf-e2b4f7d20585"
/> | <img width="1456" height="829"
alt="restore-unit-5-after-fix-reload-dropdown-open"
src="https://github.com/user-attachments/assets/7531b109-a3e5-4b2d-a8c7-745997e9c559"
/> |
| No unit selected; the run button reads "Computing Unit". | "My
Computing Unit" is selected; the run button reads "Run". |

**Change.** Both branches of `selectFromLastExecution` now go through a
new `selectFromLoadedUnits`, which waits for the first non-empty unit
list, as `selectInitialUnit` already does for a remembered unit. It then
selects the last execution's unit if it is listed, else the first
Running unit the user has WRITE access to, else nothing. The unit list
includes units shared with READ access, and the menu disables Run on
those, so falling back to one would select a unit the workflow cannot
run on; the fallback skips them. The last execution's unit is not
access-filtered: the workflow ran on it before, so it is restored as is.
The wait for the loaded list is shared between the remembered-unit
branch and the new fallback through a small `onceUnitsLoaded` helper,
which also carries the guard that drops a decision once the workflow has
changed underneath it. The warehouse preselect is unchanged.

Not in scope:

- Whether a unit that is still listed but has turned Failed should be
restored. It is selected as before.
- The check uses the unit list the page already holds. In the workspace
that list is refreshed on events such as opening the dropdown, renaming
or terminating a unit, or a websocket status change, not on a timer, so
a unit terminated in another tab since the last refresh can still be
selected.

### Any related issues, documentation, discussions?

Closes apache#8933

The last-execution path dates from apache#3400. The remembered-unit path,
which already checks the list, came with apache#8456.

### How was this PR tested?

New tests in
`frontend/src/app/workspace/component/power-button/computing-unit-selection.component.spec.ts`:

- falls back to a Running unit when the last execution's unit is no
longer listed
- skips a Running unit the user cannot run on when falling back
- selects nothing when the only Running unit is shared with READ access
- still selects the last execution's unit when it is listed with READ
access
- selects nothing when the last execution's unit is gone and no unit is
Running
- still selects the last execution's unit when it is listed but not
Running
- waits for the unit list before deciding from the latest execution
- decides from the latest execution only once, on the first loaded list
- still falls back to a Running unit when the list arrives after the
failed lookup
- drops a latest-execution decision when the list arrives after the
workflow changed
- drops a pending latest-execution decision once the component is
destroyed
- falls back to a Running unit when the latest execution has no unit
- falls back to a Running unit when both the remembered and the last
execution's unit are gone
- picks the first Running unit in list order when several are Running
- does not fall back to a unit that is Pending, Terminating, Failed or
Unknown, one case per status
- only preselects the warehouse when the unit was already settled by a
remembered choice

`makeComputingUnit` in the spec now takes an `accessPrivilege` override,
defaulting to WRITE, so the existing fixtures are unchanged.

Existing tests in the same `describe` that go through the
latest-execution path now pass their unit list to
`bootWithMetaStream(units)`, since the choice depends on the loaded
list. Their expectations are unchanged. "prefers the remembered unit for
this workflow over the latest execution" now also puts the latest
execution's unit in the list; otherwise the new code would skip that
unit anyway, and the test would no longer show that the remembered unit
wins. The existing remembered-unit tests, including "drops a pending
remembered decision once the workflow has changed underneath it", now go
through the shared helper and pass unchanged.

```
cd frontend
npx ng test --watch=false --include src/app/workspace/component/power-button/computing-unit-selection.component.spec.ts
# 214 passed
npx ng test --watch=false
# 233 files, 6596 passed | 1 skipped
npx tsc --noEmit -p tsconfig.json               # clean
npx eslint <changed files>                      # clean
npx prettier --check <changed files>            # clean
```

On Node 25 the spec run also needs
`NODE_OPTIONS="--no-experimental-webstorage"`, since its built-in
`localStorage` shadows jsdom's. CI runs Node 24 with `yarn --cwd
frontend run test:ci` and needs no flag.

Mutation check: each of these makes at least one new test fail: dropping
the list check, skipping a latest execution that has no unit, picking
the last Running unit instead of the first, picking the first unit
regardless of status, accepting any unit that is not Pending, dropping
the WRITE check from the fallback, access-filtering the last execution's
unit as well, dropping `untilDestroyed`, dropping the workflow-change
guard in the shared helper (caught from both the remembered and the
latest-execution path), dropping the non-empty-list wait, dropping
`take(1)`, reading the list synchronously in the no-history branch,
ignoring `selectUnit = false`, and preferring a Running unit over the
last execution's unit.

Manual check in Chrome, 1512x861 viewport, against a build of this
branch on the local-dev stack with local computing units:

1. Create two local units and open a workflow with one Text Input
operator.
2. Select unit A, select a warehouse, click Run, and wait for it to
complete.
3. Terminate unit A from the unit dropdown's trash icon.
4. Reload the workflow. Before: no unit is selected and the run button
reads "Computing Unit". After: the Running unit is selected and the run
button reads "Run".

Also checked by hand: when the last execution's unit is still listed,
that unit is selected rather than the first Running one; a workflow that
has never run selects the first Running unit.

### Was this PR authored or co-authored using generative AI tooling?

Yes. Generated-by: Claude Code (claude-fable-5-1, claude-opus-5-5).
Co-authored with Claude; the author reviewed the change before
submission.

---------

Co-authored-by: Kunwoo (Chris) <[email protected]>