CI: Intel SDE downloads are failing

Just opening to have an issue, but currently the two jobs:
intel_sde_avx512 and intel_sde_spr are both failing because intels download server started to reject curl...

Not sure if there is a nice work-around (to get around whatever weird check intel has, or an alternative place to fetch it from).
Even something like this: https://github.com/petarpetrovt/setup-sde except that I am always not so sure about adding "random" actions.

That is e.g. here: https://github.com/numpy/numpy/actions/runs/27258956378/job/80500546634?pr=31584

which uses:

  SDE_URL="https://downloadmirror.intel.com/859732/sde-external-9.58.0-2025-06-16-lin.tar.xz"
  SDE_SHA256="f849acecad4c9b108259c643b2688fd65c35723cd23368abe5dd64b917cc18c0"
  curl -o /tmp/sde.tar.xz "$SDE_URL"

My first thought was trying to change the user agent, e.g. curl -A "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chromium/130.0.0.0 Chrome/130.0.0.0 Safari/537.36" -o /tmp/sde.tar.xz "$SDE_URL"
That didn't work.
However, I navigated to the download page for the Intel SDE and none of the links to download the tarball worked either. I think it's a problem on the website itself.

Oh, interesting, I swear that link worked yesterday for me a couple of times. Maybe they have some very low usage limits...
(Maybe the other link if the second job uses a different one.)

We talked about this at the triage meeting and the idea of hosting the binary ourselves came up. Is the OK from a license perspective?

It works from the browser for me today, but still not from curl. From the looks of the request there's a challenge being sent back, which the browser can satisfy, but curl can't.

$ curl -v -L -A "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chromium/130.0.0.0 Chrome/130.0.0.0 Safari/537.36" -o sde.tar.xz "$SDE_URL"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Host downloadmirror.intel.com:443 was resolved.
* IPv6: (none)
* IPv4: 3.175.115.101, 3.175.115.2, 3.175.115.119, 3.175.115.92
*   Trying 3.175.115.101:443...
* Connected to downloadmirror.intel.com (3.175.115.101) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
} [329 bytes data]
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* (304) (IN), TLS handshake, Unknown (8):
{ [19 bytes data]
* (304) (IN), TLS handshake, Certificate (11):
{ [4878 bytes data]
* (304) (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* (304) (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* (304) (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: C=US; ST=California; O=Intel Corporation; CN=downloadmirror.intel.com
*  start date: May  5 00:00:00 2026 GMT
*  expire date: Aug  3 23:59:59 2026 GMT
*  subjectAltName: host "downloadmirror.intel.com" matched cert's "downloadmirror.intel.com"
*  issuer: C=GB; O=Sectigo Limited; CN=Sectigo Public Server Authentication CA OV R36
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://downloadmirror.intel.com/859732/sde-external-9.58.0-2025-06-16-lin.tar.xz
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: downloadmirror.intel.com]
* [HTTP/2] [1] [:path: /859732/sde-external-9.58.0-2025-06-16-lin.tar.xz]
* [HTTP/2] [1] [user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chromium/130.0.0.0 Chrome/130.0.0.0 Safari/537.36]
* [HTTP/2] [1] [accept: */*]
> GET /859732/sde-external-9.58.0-2025-06-16-lin.tar.xz HTTP/2
> Host: downloadmirror.intel.com
> User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chromium/130.0.0.0 Chrome/130.0.0.0 Safari/537.36
> Accept: */*
> 
* Request completely sent off
< HTTP/2 202 
< server: CloudFront
< date: Wed, 10 Jun 2026 23:36:11 GMT
< content-length: 0
< x-amzn-waf-action: challenge
< cache-control: no-store, max-age=0
< content-type: text/html; charset=UTF-8
< access-control-allow-origin: *
< access-control-max-age: 86400
< access-control-allow-methods: OPTIONS,GET,POST
< access-control-expose-headers: x-amzn-waf-action
< x-cache: Error from cloudfront
< via: 1.1 4d4befe10b69aac25e127b8999e718d2.cloudfront.net (CloudFront)
< x-amz-cf-pop: SYD3-P3
< x-amz-cf-id: bu3UH1WHh_qba2PJCegv0KBm3RVBTpBJoNTAOSfBNC6G47MUzLVQOg==
< 
{ [0 bytes data]
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Connection #0 to host downloadmirror.intel.com left intact

Anyone know why the behavior suddenly changed?

I'm pretty sure that there's a javascript challenge that gets issued when the download URL is clicked. A brower can handle this, but curl can't. I guess we could speculate on a range of reasons.
I think the options are:

Use the actions mentioned above
Cache the download somewhere, but that might depend on licence conditions
Use a tool like playwright to do a headless browser download.

I reported it to an Intel community board: https://community.intel.com/t5/Intel-ISA-Extensions/Can-t-download-Intel-SDE/m-p/1751403/emcs_t/S2h8ZW1haWx8bWVudGlvbl9zdWJzY3JpcHRpb258TVFGOTJONUZDNDM3SFJ8MTc1MTQwM3xBVF9NRU5USU9OU3xoSw#M7185

Have we ever looked at using the setup-sde github action?

https://github.com/marketplace/actions/setup-sde

I saw it originally, just am not exactly sure how many somewhat random actions we want to pull in... (It's easy to check correctness, but are we when dependabot asks us to update?)

Maybe instead we can download the binaries from their releases?

At this point... maybe that or even just use it. I had hoped this might be transient but it's been dragging on for a while.

FWIW it looks like the action has been around for a few years now. I'll open a PR.

I opened #31727 and it seems to be working.

Navigation Menu

Repository navigation

CI: Intel SDE downloads are failing #31591

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

CI: Intel SDE downloads are failing #31591

Description

Activity

andyfaff commented on Jun 10, 2026

seberg commented on Jun 10, 2026

ngoldbaum commented on Jun 10, 2026

andyfaff commented on Jun 10, 2026

r-devulap commented on Jun 11, 2026

andyfaff commented on Jun 11, 2026

andyfaff commented on Jun 16, 2026

ngoldbaum commented on Jun 23, 2026

seberg commented on Jun 23, 2026

ngoldbaum commented on Jun 23, 2026

seberg commented on Jun 23, 2026

ngoldbaum commented on Jun 23, 2026

ngoldbaum commented on Jun 23, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions