Skip to content

Add script to verify that all artefacts are in svn #14777

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
potiuk merged 2 commits into from
Mar 14, 2021
Merged

Add script to verify that all artefacts are in svn #14777

potiuk merged 2 commits into from
Mar 14, 2021

Conversation

@turbaszek
Copy link
Member

@turbaszek turbaszek commented Mar 14, 2021 •
edited
Loading

This change adds simple tool to verify that all expected files
are present in airflow svn when doing release. Also in case
of providers/backport releases it generates simple dockerfile
that can be used to verify installation.

Backports

➜ python check_files.py -v 2021.3.17rc1 -t backports -p ~/code/airflow_svn
All sources: apache-airflow-backport-providers-2021.3.17rc1
    - sources .*\.asc$: OK
    - sources .*\.sha512$: OK
    - sources .*\.(whl|gz)$: OK

apache.cassandra
    - sources .*\.asc$: OK
    - sources .*\.sha512$: OK
    - sources .*\.(whl|gz)$: OK
    - wheel .*\.asc$: OK
    - wheel .*\.sha512$: OK
    - wheel .*\.(whl|gz)$: OK

...

zendesk
    - sources .*\.asc$: OK
    - sources .*\.sha512$: OK
    - sources .*\.(whl|gz)$: OK
    - wheel .*\.asc$: OK
    - wheel .*\.sha512$: OK
    - wheel .*\.(whl|gz)$: OK

To check installation run:
        docker build -f Dockerfile.pmc -t local/airflow .
        docker run local/airflow info

FROM apache/airflow:1.10.14

# Install backports
RUN pip install 'apache-airflow-backport-providers-apache-cassandra==2021.3.17rc1'
...
RUN pip install 'apache-airflow-backport-providers-zendesk==2021.3.17rc1'

Providers

➜ python check_files.py -v 1.0.3rc1 -t providers -p ~/code/airflow_svn
elasticsearch
    - sources .*\.asc$: OK
    - sources .*\.sha512$: OK
    - sources .*\.(whl|gz)$: OK
    - wheel .*\.asc$: OK
    - wheel .*\.sha512$: OK
    - wheel .*\.(whl|gz)$: OK

To check installation run:
        docker build -f Dockerfile.pmc -t local/airflow .
        docker run local/airflow info

FROM apache/airflow:latest

# Install providers
RUN pip install 'apache-airflow-providers-elasticsearch==1.0.3rc1'

Airflow

➜ python check_files.py -v 1.10.15rc1 -t airflow -p ~/code/airflow_svn
apache_airflow-1.10.15rc1
    - binaries .*\.asc$: OK
    - binaries .*\.sha512$: OK
    - binaries .*\.(whl|gz)$: OK
    - sources .*\.asc$: OK
    - sources .*\.sha512$: OK
    - sources .*\.(whl|gz)$: OK
    - wheel .*\.asc$: OK
    - wheel .*\.sha512$: OK
    - wheel .*\.(whl|gz)$: OK

To check installation run:
        docker build -f Dockerfile.pmc -t local/airflow .
        docker run local/airflow info

FROM apache/airflow:1.10.14

# Upgrade
RUN pip install "apache-airflow==1.10.15rc1"

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@turbaszek turbaszek requested review from kaxil and potiuk March 14, 2021 14:00
@turbaszek turbaszek requested a review from ashb as a code owner March 14, 2021 14:00
@turbaszek
Add script to verify all is in svn
288cb40 
This change adds simple tool to verify that all expected files
are present in airflow svn when doing release. Also in case
of providers/backport releases it generates simple dockerfile
that can be used to verify installation.
@turbaszek turbaszek force-pushed the check-packages-are-ok branch from 64346b4 to 288cb40 Compare March 14, 2021 14:00
@turbaszek turbaszek changed the title Add script to verify all is in svn Add script to verify that all artefacts are in svn Mar 14, 2021
@potiuk
Copy link
Member

potiuk commented Mar 14, 2021

Nice. Should we also add it to the "verify by PMC" section of the dev/README ? Otherwise it might not be easy to discover.

@github-actions
Copy link

github-actions bot commented Mar 14, 2021

The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest master or amend the last commit of the PR, and push it with --force-with-lease.

@github-actions github-actions bot added the okay to merge It's ok to merge this PR as it does not require more tests label Mar 14, 2021
@turbaszek turbaszek requested a review from potiuk March 14, 2021 16:08
@potiuk potiuk merged commit 23897a5 into apache:master Mar 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@potiuk potiuk potiuk approved these changes

@kaxil kaxil kaxil approved these changes

@ashb ashb Awaiting requested review from ashb

Assignees

No one assigned

Labels

area:dev-tools okay to merge It's ok to merge this PR as it does not require more tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@turbaszek @potiuk @kaxil