Skip to content

Multi-bundle restore: support installing primitives from multiple GitHub Apps in one agent job #29

@danielmeppiel

Description

@danielmeppiel

Multi-bundle restore: support installing primitives from multiple GitHub Apps in one agent job

Background — the user's actual goal

A growing number of APM consumers run agentic workflows that need primitives drawn from more than one GitHub organization, where each org's private packages are gated behind a different GitHub App (one App per owner — that is the GitHub model).

Concretely, a single workflow needs to mix:

Today this is impossible to express cleanly with apm-action + the shared workflow pattern. This issue explains why and proposes the smallest possible action-side change to unblock it.

Why "just install them all in one job" doesn't work

Three hard constraints stack up in GitHub Actions itself.

Constraint 1 — actions/create-github-app-token is single-owner

   [email protected]
   +----------------------+
   | app-id:    ACME_APP  |  --->  ONE installation token
   | owner:     acme-org  |        scoped to acme-org ONLY
   | private-key: <key>   |
   +----------------------+

   To get a beta-org token, you MUST call it again with beta-org's app.
   No "multi-owner" mode exists in the action or the underlying API.

So: N orgs ==> N separate token mints.

Constraint 2 — gh-aw merges imported jobs by name

The natural fix is "import shared/apm.md twice with different credentials":

imports:
  - uses: shared/apm.md
    with: { app-id: ACME, owner: acme-org }
  - uses: shared/apm.md
    with: { app-id: BETA, owner: beta-org }   # COLLISION

shared/apm.md defines jobs.apm:. gh-aw merges by job key, so the second import overwrites the first. (Reference: pkg/parser/import_field_extractor.go:360-363 in microsoft/gh-aw.)

So: the shared workflow can only be imported once. Fan-out has to happen inside it.

Constraint 3 — matrix replicas are isolated runners

The GA-canonical way to run the same job N times with different parameters is strategy.matrix. That is what we do — one shared workflow with the apm job replicated per credential group:

                    apm-prep job (normalizes inputs)
                         |
            +------------+------------+
            v            v            v
       +--------+   +--------+   +--------+
       |  apm   |   |  apm   |   |  apm   |
       | public |   |  acme  |   |  beta  |
       | runner |   | runner |   | runner |
       +--------+   +--------+   +--------+
       (different) (different) (different)
        machine!    machine!    machine!

Each matrix replica is its own GA runner with its own ephemeral filesystem. They cannot share apm_modules/ directly. There is no "merge all matrix workspaces" primitive in GitHub Actions.

The forced architecture

Combine the three constraints and the only viable shape is:

+-----------------------------------------------------------------------+
|                          SHARED WORKFLOW                              |
|                                                                       |
|  apm-prep job                                                         |
|  +---------------------------------------------------+                |
|  | Input from caller:                                |                |
|  |   packages: [microsoft/apm-sample]                |                |
|  |   apps: [{owner:acme,...}, {owner:beta,...}]      |                |
|  | Normalize -> matrix manifest of N "groups"        |                |
|  +---------------------------------------------------+                |
|                              |                                        |
|                  emits matrix.group = [public, acme, beta]            |
|                              |                                        |
|                              v                                        |
|  apm job (strategy.matrix.group)                                      |
|                                                                       |
|   +--- runner #1 ----+  +--- runner #2 ----+  +--- runner #3 ----+    |
|   | group: public    |  | group: acme      |  | group: beta      |    |
|   | no token mint    |  | mint ACME token  |  | mint BETA token  |    |
|   |       |          |  |       |          |  |       |          |    |
|   |       v          |  |       v          |  |       v          |    |
|   | apm-action PACK: |  | apm-action PACK: |  | apm-action PACK: |    |
|   | install + tar.gz |  | install + tar.gz |  | install + tar.gz |    |
|   |       |          |  |       |          |  |       |          |    |
|   |       v          |  |       v          |  |       v          |    |
|   | upload-artifact: |  | upload-artifact: |  | upload-artifact: |    |
|   | "apm-public"     |  | "apm-acme"       |  | "apm-beta"       |    |
|   +------------------+  +------------------+  +------------------+    |
|                              |                                        |
|         (3 separate tarballs sitting in GA artifact storage)          |
|                              |                                        |
|                              v                                        |
|  agent job                                                            |
|   +--------------------------------------------------+                |
|   | pre-agent-steps:                                 |                |
|   |   download-artifact pattern: apm-*               |                |
|   |   +-------------+ +-------------+ +-------------+|                |
|   |   |apm-public.gz| | apm-acme.gz | | apm-beta.gz ||                |
|   |   +-------------+ +-------------+ +-------------+|                |
|   |                          |                       |                |
|   |                          v                       |                |
|   |   <======= THIS IS THE GAP =======>              |                |
|   |   apm-action restore: needs to merge all 3 into  |                |
|   |   the SAME workspace before the AI agent runs    |                |
|   +--------------------------------------------------+                |
|                              |                                        |
|                              v                                        |
|              copilot/claude/codex runs the agent                      |
|              with merged primitives from all 3 orgs                   |
+-----------------------------------------------------------------------+

Where this issue fits

Today apm-action accepts:

bundle: /tmp/foo.tar.gz       # restores ONE bundle

The agent job needs to feed it N bundles in one call, because:

  • A uses: step cannot itself be matrixed (matrix is job-level only in GA).
  • Calling apm-action N times sequentially in N statically-written steps would defeat the purpose of a shared workflow — the caller doesn't know N at write time (it is length(apps) resolved at runtime).

So we need:

bundles-file: /tmp/gh-aw/apm-bundle-list.txt   # restores N bundles in order

Where the list file is just newline-separated bundle paths:

/tmp/gh-aw/apm-bundles/apm-public/build/bundle.tar.gz
/tmp/gh-aw/apm-bundles/apm-acme/build/bundle.tar.gz
/tmp/gh-aw/apm-bundles/apm-beta/build/bundle.tar.gz

apm-action loops internally, calling apm unpack once per bundle into the same workspace. That is the entire feature.

The chain of "why not just X?"

Question Why not
Why not install everything in one job? Tokens are single-owner (Constraint 1)
Why not import shared/apm.md N times? Job-name collision (Constraint 2)
Why not have one apm job loop over all orgs? Need N tokens minted dynamically -> 50+ lines of dynamic YAML; also loses parallelism
Why not have matrix replicas write to a shared volume? GA matrix replicas are isolated runners (Constraint 3) — no shared FS
Why not concatenate all bundles into one .tar.gz before restore? Would need a dedicated 'merge' job that knows APM's bundle format (lockfile, deployed_files manifest, security gate) — that logic ALREADY exists inside apm unpack, just call it N times
Why not call apm-action N times sequentially in pre-agent-steps? Caller doesn't know N at write time (it is a runtime property of apps[] length); would force every caller to write a bash loop

Summary

bundles-file: is the smallest possible action-side change that closes the loop:

Workflow side:    fan-out (matrix)  ->  upload N artifacts
                                              |
                                              v
                                     download N artifacts
                                              |
                                              v
Action side:               <----- HERE — restore N bundles ----->
                                              |
                                              v
                                     single merged workspace
                                              |
                                              v
                                     agent runs with all primitives

Without bundles-file:, the matrix output (N artifacts) has nowhere to land. The workflow can mint the tokens, install per-org, upload the bundles — but the agent job cannot reassemble them. It is literally the last mile.

Cross-references

A detailed technical design (input contract, error semantics, security requirements, test plan) will be posted as the body of the implementing PR in this repo.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions