Skip to content

Comments

Fix deployment update worker queues#1886

Merged
feluelle merged 1 commit intomainfrom
fix/deployment-update-worker-queues
Jul 1, 2025
Merged

Fix deployment update worker queues#1886
feluelle merged 1 commit intomainfrom
fix/deployment-update-worker-queues

Conversation

@feluelle
Copy link
Member

@feluelle feluelle commented Jul 1, 2025 •
edited
Loading

Description

Describe the purpose of this pull request.

updateWorkerQueuesForExecutor used for deployment update currently overwrites the worker queues if the worker queues are not set. However, we want the opposite. We want to use the worker queues request if set, and if not, use the default.

🎟 Issue(s)

Related #XXX

🧪 Functional Testing

List the functional testing steps to confirm this feature or fix.

Inspect a deployment with worker queues

❯ ~/Projects/astro-cli/astro deployment inspect
Select a Deployment
 #     DEPLOYMENT NAME     RELEASE NAME         DEPLOYMENT ID                 DAG DEPLOY ENABLED     
 1     test                modern-beam-2288     cmckmn35w0er001l4fw8v8136     true                   

> 1
deployment:
    ...
    worker_queues:
        - name: default
          min_worker_count: 0
          worker_type: A5
          pod_cpu: "1"
          pod_ram: 2Gi
        - name: foo
          min_worker_count: 0
          worker_type: A5
          pod_cpu: "1"
          pod_ram: 2Gi
...

Update the deployment

❯ ~/Projects/astro-cli/astro deployment update
Select a Deployment
 #     DEPLOYMENT NAME     RELEASE NAME         DEPLOYMENT ID                 DAG DEPLOY ENABLED     
 1     test                modern-beam-2288     cmckmn35w0er001l4fw8v8136     true                   

> 1
 NAME     NAMESPACE            CLUSTER                  CLOUD PROVIDER     REGION      DEPLOYMENT ID                 RUNTIME VERSION                    DAG DEPLOY ENABLED     CI-CD ENFORCEMENT     DEPLOYMENT TYPE     
 test     modern-beam-2288     airflow3-testing_DND     AZURE              westus2     cmckmn35w0er001l4fw8v8136     3.0-4 (based on Airflow 3.0.2)     true                   false                 DEDICATED           

 Successfully updated Deployment

Inspect the deployment

❯ ~/Projects/astro-cli/astro deployment inspect
Select a Deployment
 #     DEPLOYMENT NAME     RELEASE NAME         DEPLOYMENT ID                 DAG DEPLOY ENABLED     
 1     test                modern-beam-2288     cmckmn35w0er001l4fw8v8136     true                   

> 1
deployment:
    ...
    worker_queues:
        - name: default
          min_worker_count: 0
          worker_type: A5
          pod_cpu: "1"
          pod_ram: 2Gi
        - name: foo
          min_worker_count: 0
          worker_type: A5
          pod_cpu: "1"
          pod_ram: 2Gi
...

📸 Screenshots

Add screenshots to illustrate the validity of these changes.

📋 Checklist

  • Rebased from the main (or release if patching) branch (before testing)
  • Ran make test before taking out of draft
  • Ran make lint before taking out of draft
  • Added/updated applicable tests
  • Tested against Astro-API (if necessary).
  • Tested against Houston-API and Astronomer (if necessary).
  • Communicated to/tagged owners of respective clients potentially impacted by these changes.
  • Updated any related documentation

@feluelle
Fix deployment update worker queues
9a72ffd 
`updateWorkerQueuesForExecutor` used for deployment update currently overwrites the worker queues if the worker queues are not set. However, we want the opposite. We want to use the worker queues request if set, and if not, use the default.
@feluelle feluelle requested a review from a team July 1, 2025 16:00
@feluelle feluelle mentioned this pull request Jul 1, 2025
Merged
8 tasks
@feluelle feluelle self-assigned this Jul 1, 2025
@feluelle feluelle merged commit 0b680ad into main Jul 1, 2025
4 of 5 checks passed
@feluelle feluelle deleted the fix/deployment-update-worker-queues branch July 1, 2025 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jaketf jaketf jaketf approved these changes

@neel-astro neel-astro Awaiting requested review from neel-astro

@jeremybeard jeremybeard Awaiting requested review from jeremybeard

@kushalmalani kushalmalani Awaiting requested review from kushalmalani

Assignees

@feluelle feluelle

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@feluelle @jaketf