How to use env vars or constants in composite actions? #51280

mildmojo · 2023-03-28T23:05:25Z

mildmojo
Mar 28, 2023

Select Topic Area

Question

Body

When I write a reusable workflow, I can set environment variables that are available to the action templating engine under a top-level env key, like:

# .github/workflows/docker-build.yml
name: "Docker Build"

on:
  workflow_call:
    inputs:
      tag_name:
        description: "Docker tag to publish"
        type: string
        required: false

env:
  IMAGE_TAG: "my-docker-registry.example.com/my-images:${{ inputs.tag }}"

jobs:
  docker_build:
    runs-on: [self-hosted]
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
  
      - name: Build Image
        uses: docker/build-push-action@v3
        with:
          tags: ${{ env.IMAGE_TAG }}

However, I'm struggling to figure out how to do this in a composite action. Putting an env map at the top level, or under the job definition as a sibling of steps: seems to throw syntax errors. This seems to fail to reference the inputs object:

# /docker-build/action.yml
name: "Docker Build"

inputs:
  tag_name:
    description: "Docker tag to publish"
    type: string
    required: false

env:
  IMAGE_TAG: "my-docker-registry.example.com/my-images:${{ inputs.tag }}"

runs:
  using: "composite"
  steps:
    - name: Checkout repository
      uses: actions/checkout@v3

    - name: Build Image
      uses: docker/build-push-action@v3
      with:
        tags: ${{ env.IMAGE_TAG }}

Error: my-org/my-actions/main/docker-build/action.yml (Line: 51, Col: 30):
Error: my-org/my-actions/main/docker-build/action.yml (Line: 52, Col: 25):
Error: my-org/my-actions/main/docker-build/action.yml (Line: 51, Col: 30): Unrecognized named-value: 'inputs'. Located at position 1 within expression: inputs.tag
Error: my-org/my-actions/main/docker-build/action.yml (Line: 52, Col: 25): Unrecognized named-value: 'inputs'. Located at position 1 within expression: inputs.tag
Error: GitHub.DistributedTask.ObjectTemplating.TemplateValidationException: The template is not valid. my-org/my-actions/main/docker-build/action.yml (Line: 51, Col: 30): Unrecognized named-value: 'inputs'. Located at position 1 within expression: inputs.tag,my-org/my-actions/main/docker-build/action.yml (Line: 52, Col: 25): Unrecognized named-value: 'inputs'. Located at position 1 within expression: inputs.tag

Moving env under runs also causes parsing errors:

# /docker-build/action.yml
...
runs:
  using: "composite"
  env:
    IMAGE_TAG: "my-docker-registry.example.com/my-images:${{ inputs.tag }}"
  steps:
    - name: Checkout repository
      uses: actions/checkout@v3

    - name: Build Image
      uses: docker/build-push-action@v3
      with:
        tags: ${{ env.IMAGE_TAG }}

Error: my-org/my-actions/main/docker-build/action.yml (Line: 55, Col: 3): Unexpected value 'steps'
Error: my-org/my-actions/main/docker-build/action.yml (Line: 55, Col: 3): Unexpected value 'steps'
Error: System.ArgumentNullException: Value cannot be null. (Parameter 'You are using a composite action but there are no steps provided in my-org/my-actions/main/docker-build/action.yml.')

Is this possible? Do I have a syntax error? Can I build template variables in a composite action? Or do I have to write those values out longhand everywhere I want to use them?

Note that I'm not trying to set shell environment variables. I need variables that I can pass as inputs to other actions in with: blocks, not stuff I can only reference in runs/shell steps.

Thanks!

Answered by mildmojo

Mar 9, 2024

Answer

There's no top-level YAML way to declare a variable scoped to a composite action and available in step options, like you can with workflows and the top-level env. 😔 Maybe they'll add it someday! See a workaround below.

Docker actions do have a runs.env that lets you declare variables at a top-ish level, but those aren't composite actions, and I haven't tested to see if that modifies the caller's env.

Thanks to everyone who replied!

Details

Here's what I learned from the replies, docs, and my own testing:

A workflow's top-level env, jobs.<id>.env, and the space you manipulate by writing to $GITHUB_ENV in a shell command (in both workflows and actions) are all the same globally-shar…

View full answer

radiomix · 2023-05-16T15:04:01Z

radiomix
May 16, 2023

@chrispat The question is valid! Setting environment variables in composite actions saves setting them in each calling workflow.

0 replies

cefeboru · 2023-05-26T17:51:32Z

cefeboru
May 26, 2023

I have the same question ✋ - Trying to set NPM_TOKEN for one of our customs actions and wanted to define it for every step

0 replies

mrafnadeem-apimatic · 2023-06-01T06:41:39Z

mrafnadeem-apimatic
Jun 1, 2023

The composite action should have the environment variable available by default if it's defined in the calling workflow. But I'm also getting the same issue regardless of whether I pass in the environment variable or not!

0 replies

joeatnetgear · 2023-06-14T07:37:05Z

joeatnetgear
Jun 14, 2023

I found a partial work around. Still seems to be a bug or at least an unfortunate design pattern.

In a step before the uses call save the environment variable to $GITHUB_ENV

 run: echo "GR=$GITHUB_REF"  >> $GITHUB_ENV

In a subsequent step you can pass it as one of the variables named using the with parameter

uses: ./actions/12GitBranchCA
      with: 
         ref: ${{ env.GR }}

Then in the action's action.yml of the composite script you can use it as any
other input parameter.

name: "test1 get git branch"
 inputs:
     ref:
       type: string
       required: true

 outputs:
       branch: 
         value: ${{ steps.step1.outputs.branch }}

 runs:
    using: "composite"
    steps:
    - name: test2
      run: echo "GITHUB_REF=${{inputs.ref}}"
      shell: bash
    - id: step1
      name: get git branch 
      run: echo "branch=$(echo ${{inputs.ref}} | cut -d'/' -f 3)"  >> $GITHUB_OUTPUT
      shell: bash

I swear this YML syntax for what should have been a simple function call is quite silly. They would have been much more efficient just starting with Python or Lua as a embedded scripting language with strong supporting libraries rather than reinventing the wheel in such a convoluted fashion. Even the old perl syntax for includes or requires is cleaner. Even ADA is less verbose and easier to read.

1 reply

mildmojo Jun 14, 2023
Author

This is interesting; I didn't realize you could refer to env vars created in a run step with echo "VAR=5" >> $GITHUB_ENV using template syntax ${{ env.VAR }}. I figured template syntax was interpolated before the action began, didn't realize it was dynamic. It's a little noisy, but if this works, I guess it's fine putting all my template constants as $GITHUB_ENV vars in a single run step at the top. I'll try it out.

laschoe · 2023-06-14T11:11:53Z

laschoe
Jun 14, 2023

I found a working workaround and I did as suggested here:
https://stackoverflow.com/questions/73797254/environment-variables-in-github-actions

The workaround from @joeatnetgear did not work for me.

3 replies

joeatnetgear Jun 14, 2023

Can you post your YML I am curious about why it did not work.

laschoe Jun 21, 2023

Sorry, I was not able to recreate the error, but mainly the problem was that GitHub told me: 'unrecognized named-value 'env'' after echoing a variable to the environment in a previous step.

leemeador Oct 28, 2023

@lschoenepxc Your link to stack overflow is regarding a workflow that runs when an event occurs. The rules for "env" seem to be different. The OP is asking about the contents of action.yaml in a repo used as a GitHub Action. This is an "action" as opposed to a "workflow" the way I talk about it. Its unfortunate that GitHub used the terminology such that people end up calling every workflow an action.

leemeador · 2023-10-28T02:05:07Z

leemeador
Oct 28, 2023

@mildmojo
In your first try, the error is saying the 'inputs' are not available at that point in the yaml file. It's not that 'env' isn't allowed at that point. I don't think it sets env variables but that's not the problem you experienced.

In your 2nd try, the error is that "env" is not allowed under a "runs" in a composite action. The docs are here https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runs-for-composite-actions and following. As you can see looking down through there, 'env' is only allowed under one of the steps (not step itself). When 'env' is under a step, it only applies to that step.

This is the doc for that https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runsstepsenv but notice that it says " If you want to modify the environment variable stored in the workflow, use echo "{name}={value}" >> $GITHUB_ENV in a composite step."

When you do that, the env should be set starting in the next step after that "echo ..." and all subsequent steps.

3 replies

LesnyRumcajs Jan 16, 2024

Worked as a charm, thank you!

TWiStErRob Feb 25, 2024

Mind you that this affects the caller! So only set variables that don't mess with the user, or clean up afterwards.

mildmojo Mar 8, 2024
Author

Thanks for the reply, leemeador! But yes, you've restated my question, including my observations about what doesn't work and why.

Thanks for linking to those docs, which seem to definitively answer that no, there isn't a way to set a variable scoped to the action. It's frustrating because runs.env exists for Docker actions.

I'll update my question with a summary of the stuff everyone's noted, and what I've tried.

mildmojo · 2024-03-09T01:01:07Z

mildmojo
Mar 9, 2024
Author

Answer

There's no top-level YAML way to declare a variable scoped to a composite action and available in step options, like you can with workflows and the top-level env. 😔 Maybe they'll add it someday! See a workaround below.

Docker actions do have a runs.env that lets you declare variables at a top-ish level, but those aren't composite actions, and I haven't tested to see if that modifies the caller's env.

Thanks to everyone who replied!

Details

Here's what I learned from the replies, docs, and my own testing:

A workflow's top-level env, jobs.<id>.env, and the space you manipulate by writing to $GITHUB_ENV in a shell command (in both workflows and actions) are all the same globally-shared storage, and are bidirectionally shared between a calling workflow and a composite action. Setting env.MY_VARin the caller sets it for the callee, and vice versa. It's all one global key-value store for a workflow run and its entire call graph of other workflows and actions. It's automatically copied into shell environments for individual steps, but setting shell environment vars inside a shell step doesn't modify the global env; you have to write to $GITHUB_ENV for that. Thanks to @radiomix, @joeatnetgear, @mrafnadeem-apimatic, @leemeador, and @TWiStErRob for alluding to this.
Composite action YAML syntax has no support for an env key anywhere; that's reserved for workflows and Docker actions.
GitHub action YAML does not support YAML anchors, which would otherwise be useful if you wanted to copy a value wholesale without any string manipulation.
You can use a shell step in a composite action to write to $GITHUB_ENV, which will make that named value available to use with ${{ env.MY_VAR }} syntax in other action step parameters, even ones like steps.shell. However, because that key-value store is globally-shared, this will modify it upstream in the calling workflow or action, too, which may not be obvious and may cause weird bugs. You'd want to heavily prefix your composite action env var names to avoid conflicts, especially if you're publishing your action.
A better workaround below, suggested by @TWiStErRob, would be to use step outputs. These won't leak into the caller's scope or any sub-actions' scopes. They're a little more verbose, but y'know, there's a reason they're not called "playarounds".

Workaround

Using $GITHUB_OUTPUT from a shell step in a composite action may be a workaround for you. For example, a composite action with a step designed to store global variables as outputs for use in other steps:

name: "Test"
description: "Test composite actions setting global variables"

runs:
  using: "composite"
  steps:
    - name: Set globals
      id: globals
      shell: bash
      run: |
        echo "TEST_ACTION_SHELL=bash" >> "${GITHUB_OUTPUT}"
        echo "TEST_ACTION_RUN=date" >> "${GITHUB_OUTPUT}"
        echo "TEST_ACTION_NODE_VERSION=12.0.0" >> "${GITHUB_OUTPUT}"

    # Runs the `date` command in bash and outputs the date/time.
    - name: Run command
      shell: ${{ steps.globals.outputs.TEST_ACTION_SHELL }}
      run: ${{ steps.globals.outputs.TEST_ACTION_RUN }}

    - uses: actions/setup-node@v4
      with:
        node-version: ${{ steps.globals.outputs.TEST_ACTION_NODE_VERSION }}

    # Correctly outputs v12.0.0
    - name: Check node version
      shell: bash
      run: node --version

5 replies

laschoe Mar 11, 2024

Here is also a very similiar way to use env vars "globally": build-and-test.yml

env:
  dotnet_sdk_version: '8.0.100'
  REPOSITORY_NAME: ${{ github.event.repository.name }}

jobs:
  EnvVar:
    runs-on: ubuntu-latest
    steps:
      - run: echo ""
    outputs:
      dotnet_sdk_version: ${{ env.dotnet_sdk_version }}
      REPOSITORY_NAME: ${{ env.REPOSITORY_NAME }}

  Build:
    needs: [EnvVar]
    uses: phoenixcontact/tools/.github/workflows/build-tool.yml@main
    with:
      dotnet_sdk_version: ${{ needs.EnvVar.outputs.dotnet_sdk_version }}
      REPOSITORY_NAME: ${{ needs.EnvVar.outputs.REPOSITORY_NAME }}

mildmojo Mar 18, 2024
Author

That looks like a workflow, not a composite action. And yes, top-level env is great in workflows! That's what I was trying to find/replicate in composite actions.

laschoe Mar 19, 2024

Yes, that's right. It's the other way round: setting the env vars for workflows calling composite actions. Sorry. I will leave that comment, nevertheless, in case someone might want to do the same thing in the future.

TWiStErRob Mar 19, 2024

Nice summary!

I would like to add that preferring step outputs might be a better workaround. If someone has a step to expose env vars, it's trivial to move over to step outputs and that removes the global pollution.

steps:
  - name: "Composite action-local global variables"
+   id: vars
    shell: bash
    run: |
-     echo "IMAGE-TAG=my-docker-registry.example.com/my-images:${{ inputs.tag }}" >> "${GITHUB_ENV}"
+     echo "image-tag=my-docker-registry.example.com/my-images:${{ inputs.tag }}" >> "${GITHUB_OUTPUT}"
-     echo "JAVA_HOME=$(complex.sh)" >> "${GITHUB_ENV}"
+     echo "java-home=$(complex.sh)" >> "${GITHUB_OUTPUT}"

  - name: "Some special Java step needing env var (works with uses too)"
+   env:
+     JAVA_HOME: ${{ steps.vars.outputs.java-home }}
    shell: bash
    run: ./gradlew build

  - name: Build Image
    uses: docker/build-push-action@v3
    with:
-     tags: ${{ env.IMAGE_TAG }}
+     tags: ${{ steps.vars.outputs.image-tag }}

This might be a bit of duplication if lot of the steps are using the same var(s), but it might be worth it for the scoping and explicitness in return.

mildmojo Mar 19, 2024
Author

There are a lot of different use cases expressed in the replies and comments! You're right; it makes references a little more verbose, but outputs are probably a better revision to this workaround. Thanks for this! I'll amend my suggestion.

kasir-barati · 2024-03-19T16:31:17Z

kasir-barati
Mar 19, 2024

Hey guys, why it says in docs that we can do it: https://docs.github.com/en/actions/creating-actions/creating-a-composite-action#creating-an-action-metadata-file

To me it seems like we can do it, though I have not tried it yet. Will do and share the results with you.

1 reply

mildmojo Mar 19, 2024
Author

That's a step-level env; I was looking for an env that would store values available to all steps in the composite action, like the top-level env in workflows. In particular, a value that's available for use in options passed to other actions in the YAML, not just shell interpolation. For example, I'd be really surprised if this worked, and it wouldn't be that useful anyway:

runs:
  using: composite
  steps:
    - name: Echo
      env:
        SHELL: bash
      run: echo Hello!
      shell: ${{ env.SHELL }}

gimlichael · 2024-03-29T20:22:57Z

gimlichael
Mar 29, 2024

This is a needed feature; only suitable workaround as I see it, is to use inputs. However, it would be nice with an isolated approach as you propose, as this workaround seem artificial.

0 replies

h3ct0rjs · 2024-05-06T19:50:37Z

h3ct0rjs
May 6, 2024

Hey,
I arrived here looking for an answer and it seems like the best option is using the GITHUB_OUTPUT not elegant but it is useful. This should be a feature request 😄. Can anybody mention this as a feature request for the GHA team? I don't know how Github manages this but If I understand correctly a few Github members are looking for feature requests.

0 replies

jhchill666 · 2024-07-02T17:51:18Z

jhchill666
Jul 2, 2024

Forgive me if I'm wrong, but setting 'top level' env vars in the workflow invoking the composite action, allows these env vars to be accessed in the composite action:

// some-workflow.yml

env:
  SOME_VARIABLE: "123"

jobs:
  RunJob:
    runs-on: ubuntu-latest
  
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/some-action

// some-action/action.yml

runs:
  using: "composite"
  steps:
    - run: echo ${{ env.SOME_VARIABLE }}
      env:
        SOME_VARIABLE: ${{ env.SOME_VARIABLE }}

1 reply

mildmojo Jul 2, 2024
Author

I haven't tested that, but it's essentially another way to send an input to a composite action, which isn't what I was looking for. I'd rather use regular action inputs for that.

In the examples in my original question, you can see how I was trying to build a value that's scoped to the composite action, that's owned by the composite action. In my example, I was building a docker image tag like my-docker-registry.example.com/my-images:${{ inputs.tag }}, where a partial tag is the input to the action, and the action itself hardcodes the registry and repository name, building an IMAGE_TAG variable that can be referenced in the action's steps or provided as inputs to other actions. The calling workflow wouldn't know the registry/repo names, and shouldn't have to do the work of building the full image tag string for the composite action to use.

ABOBAKAR-IT · 2024-10-03T19:37:01Z

ABOBAKAR-IT
Oct 3, 2024

on:
push:
branches:
- main

jobs:
deploy:
name: Deploy to EC2
runs-on: ubuntu-latest
env:
VITE_API_BASE_URL: ${{ secrets.VITE_API_BASE_URL }}

steps:
  - name: Checkout Repository
    uses: actions/checkout@v4

  - name: Setup Node.js
    uses: actions/setup-node@v4
    with:
      node-version: '20'
  


  - name: Install Dependencies
    run: npm install

  - name: env setup
    run: | 
      export VITE_API_BASE_URL=${{secrets.VITE_API_BASE_URL}} 

  - name: Build Project
    run: npm run build

1 reply

mildmojo Oct 3, 2024
Author

That looks like a workflow, not a composite action. I put an example like this at the top of the original post. Unfortunately, composite actions don't have an env key.

How to use env vars or constants in composite actions? #51280

Uh oh!

Select Topic Area

Body

Answer

Details

Replies: 12 comments · 15 replies

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mildmojo Jun 14, 2023 Author

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mildmojo Mar 8, 2024 Author

Uh oh!

Uh oh!

mildmojo Mar 9, 2024 Author

Answer

Details

Workaround

Uh oh!

Uh oh!

Uh oh!

mildmojo Mar 18, 2024 Author

Uh oh!

Uh oh!

Uh oh!

mildmojo Mar 19, 2024 Author

Uh oh!

Uh oh!

Uh oh!

mildmojo Mar 19, 2024 Author

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Replies: 12 comments 15 replies

mildmojo Jun 14, 2023
Author

mildmojo Mar 8, 2024
Author

mildmojo
Mar 9, 2024
Author

mildmojo Mar 18, 2024
Author

mildmojo Mar 19, 2024
Author

mildmojo Mar 19, 2024
Author