Skip to content

update to the new CDS-beta#37

Merged
yunjunz merged 3 commits intoinsarlab:mainfrom
yuankailiu:main
Sep 18, 2024
Merged

update to the new CDS-beta#37
yunjunz merged 3 commits intoinsarlab:mainfrom
yuankailiu:main

Conversation

@yuankailiu
Copy link
Copy Markdown
Contributor

@yuankailiu yuankailiu commented Sep 16, 2024

This old instance of the CDS will be decommissioned on 26 September 2024 and will no longer be accessible from this date onwards. Let's migrate the guide and the code to CDS-Beta.

NEW URL: https://cds-beta.climate.copernicus.eu/api

Commits:

  • update beta version url in README
  • update beter version url in autoget.py
  • for key within either .cdsapirc or model.cfg, only type in your personal access token, no UID is needed and allowed.
  • point to where to find my token.
  • point to where to accept the Terms of Use (since I find it hard to see the button).

Examples

My model.cfg looks like:

# old style
[CDS]
key = 311901:e041172e-0a2a-4e0f-88fa-ea3b6bae7058

# new style
[CDS]
key = ebfbd500-719b-4e03-aca7-6c880f64bf79

My ~/.cdsapirc looks like:

# old style
url: https://cds.climate.copernicus.eu/api/v2
key: 311901:e041172e-0a2a-4e0f-88fa-ea3b6bae7058

# new style
url: https://cds-beta.climate.copernicus.eu/api
key: ebfbd500-719b-4e03-aca7-6c880f64bf79

Test:

I guess you might still use this old style until 9/26 with this reminder pops up:

As per our announcements on the Forum, this instance of CDS will be decommissioned on 26 September 2024 and will no longer be accessible from this date onwards.
Please update your cdsapi package to a version >=0.7.2, create an account on CDS-Beta and update your .cdsapirc file. We strongly recommend users to check our Guidelines at https://confluence.ecmwf.int/x/uINmFw

And the request will be queued and get stuck.

Update the new url and using the token from the beta website, run the test:

cd tests/ && python test_dload.py

it works:

------------------------------------------------
import pyaps3 from /home/ykliu/apps/mambaforge/envs/insar/lib/python3.12/site-packages/pyaps3/__init__.py
------------------------------------------------
test ERA5 data download
NOTE: Account setup is required on the Copernicus Climate Data Store (CDS).
      More detailed info can be found on: https://retostauffer.org/code/Download-ERA5/
      Add your account info to ~/.cdsapirc file.
INFO: You are using the latest ECMWF platform for downloading datasets:  https://cds-beta.climate.copernicus.eu/api
Downloading 1 of 2: /home/ykliu/apps/PyAPS/tests/data/ERA5/ERA5_N30_N40_E120_E140_20200601_14.grb
{'product_type': 'reanalysis', 'format': 'grib', 'variable': ['geopotential', 'temperature', 'specific_humidity'], 'pressure_level': ['1', '2', '3', '5', '7', '10', '20', '30', '50', '70', '100', '125', '150', '175', '200', '225', '250', '300', '350', '400', '450', '500', '550', '600', '650', '700', '750', '775', '800', '825', '850', '875', '900', '925', '950', '975', '1000'], 'year': '2020', 'month': '06', 'day': '01', 'time': '14:00', 'area': '40.00/120.00/30.00/140.00'}
2024-09-16 15:44:43,153 INFO Request ID is c4bdbde8-f73c-4dfa-b92a-58006ab76355
2024-09-16 15:44:43,333 INFO status has been updated to accepted
2024-09-16 15:44:45,026 INFO status has been updated to running
2024-09-16 15:44:47,459 INFO status has been updated to successful
Downloading 2 of 2: /home/ykliu/apps/PyAPS/tests/data/ERA5/ERA5_N30_N40_E120_E140_20200901_14.grb
{'product_type': 'reanalysis', 'format': 'grib', 'variable': ['geopotential', 'temperature', 'specific_humidity'], 'pressure_level': ['1', '2', '3', '5', '7', '10', '20', '30', '50', '70', '100', '125', '150', '175', '200', '225', '250', '300', '350', '400', '450', '500', '550', '600', '650', '700', '750', '775', '800', '825', '850', '875', '900', '925', '950', '975', '1000'], 'year': '2020', 'month': '09', 'day': '01', 'time': '14:00', 'area': '40.00/120.00/30.00/140.00'}
2024-09-16 15:44:51,651 INFO Request ID is e6348440-f2c1-4e5b-9781-dfdf3d477c85
2024-09-16 15:44:51,816 INFO status has been updated to accepted
2024-09-16 15:44:53,493 INFO status has been updated to successful
------------------------------------------------
Downloads OK
------------------------------------------------

@yuankailiu
cds api beta version url and token
6f49dbf 
+ update beta version url

+ personal access token, no UID

+ where to find my token

+ where to accept the Terms of Use
@yunjunz yunjunz linked an issue Sep 18, 2024 that may be closed by this pull request
New CDS API #36
Closed
@yunjunz
update comments & links
e643c02 
+ dependency: add cdsapi>=0.7.0 version constraint on pyproject.toml and requirements.txt files

+ tests/test_dload.py: update print out website link

+ README:
   - remove obsolete link to retostauffer.org website
   - update link to the "create a new account" link
@yunjunz yunjunz changed the title cds api beta version url and token update to the new CDS-beta Sep 18, 2024
yunjunz
yunjunz approved these changes Sep 18, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@yunjunz yunjunz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much @yuankailiu! Following your note, I registered a new account, updated the ~/.cdsapirc file, and everything looks nicely with the new CDS-beta. I will go ahead and merge this, and cut a new release.

@yunjunz yunjunz merged commit 7d596cc into insarlab:main Sep 18, 2024
@zhangxing-sigma
Copy link
Copy Markdown

zhangxing-sigma commented Oct 28, 2024

I saw a reminder on the ERA5 website asking if it is necessary to update the pyaps3 file
if you are a CDS-beta API user, requests will be automatically redirected from the beta URL to the proper one until 20th October 2024. You are advised to update the URL in your .cdsapirc file at your earliest convenience by 20th October 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yunjunz yunjunz yunjunz approved these changes

@jhkennedy jhkennedy Awaiting requested review from jhkennedy

@scottstanie scottstanie Awaiting requested review from scottstanie

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

New CDS API

3 participants

@yuankailiu @zhangxing-sigma @yunjunz