Skip to content

ARROW-17423: [CI][C++] Fix building CUDA docker images - #13896

Merged
pitrou merged 3 commits into
apache:masterfrom
pitrou:ARROW-17423-cpp-cuda-ci
Aug 17, 2022
Merged

pitrou merged 3 commits into
apache:masterfrom
pitrou:ARROW-17423-cpp-cuda-ci

Conversation

@pitrou

@pitrou pitrou commented Aug 16, 2022

Copy link
Copy Markdown
Member
  • Update the CUDA runtime version as CUDA 9.1 images are not available anymore
  • Fix passing child command arguments to "docker run"

Checked locally under a Ubuntu 20.04 host with:

UBUNTU=18.04 archery --debug docker run ubuntu-cuda-cpp
UBUNTU=20.04 archery --debug docker run ubuntu-cuda-cpp

@pitrou
pitrou requested a review from kszucs August 16, 2022 12:18
@pitrou

pitrou commented Aug 16, 2022

Copy link
Copy Markdown
Member Author

cc @assignUser

@github-actions

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@assignUser assignUser left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on. Looks good with a minor change to rsplit. We should test it with some windows and linux jobs.

Comment thread dev/archery/archery/docker/core.py Resolved
Comment thread docker-compose.yml Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this is to reduce build times and focus this on CUDA? 👍 Good idea!

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

@pitrou

pitrou commented Aug 16, 2022

Copy link
Copy Markdown
Member Author

@kkraus14 Would you like to take a quick look?

Comment thread .env Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may make sense to pick the oldest supported version here instead of this so that someone doesn't accidentally use a feature introduced in a newer driver version and allow it to slip through CI.

@pitrou pitrou Aug 16, 2022 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would seem to be 11.0.3 for Ubuntu 20.04. Does it make sense @kkraus14 ?
(note that this variable will have to be overriden depending on the Ubuntu version selected, anyway)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Ubuntu 20.04 that makes sense, if we could use 18.04 I would suggest going with 10.0 instead.

I would hope that everyone is using a 11.0+ driver today though.

@kszucs kszucs left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

pitrou added 3 commits August 17, 2022 10:25
* Update the CUDA runtime version as CUDA 9.1 images are not available anymore
* Fix passing child command arguments to "docker run"

Checked locally under a Ubuntu 20.04 host with:
```
UBUNTU=18.04 archery --debug docker run ubuntu-cuda-cpp
UBUNTU=20.04 archery --debug docker run ubuntu-cuda-cpp
```
@pitrou
pitrou force-pushed the ARROW-17423-cpp-cuda-ci branch from ccaad19 to a4e051a Compare August 17, 2022 08:45
@pitrou
pitrou merged commit f6127fc into apache:master Aug 17, 2022
@pitrou
pitrou deleted the ARROW-17423-cpp-cuda-ci branch August 17, 2022 09:00
@ursabot

ursabot commented Aug 17, 2022

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = 050876c and contender = f6127fc. f6127fc is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️50.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️9.08% ⬆️7.93%] test-mac-arm
[Failed ⬇️7.4% ⬆️8.22%] ursa-i9-9960x
[Finished ⬇️9.82% ⬆️8.04%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] f6127fca ec2-t3-xlarge-us-east-2
[Finished] f6127fca test-mac-arm
[Failed] f6127fca ursa-i9-9960x
[Finished] f6127fca ursa-thinkcentre-m75q
[Finished] 050876c5 ec2-t3-xlarge-us-east-2
[Failed] 050876c5 test-mac-arm
[Failed] 050876c5 ursa-i9-9960x
[Finished] 050876c5 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@ursabot

ursabot commented Aug 17, 2022

Copy link
Copy Markdown

['Python', 'R'] benchmarks have high level of regressions.
ec2-t3-xlarge-us-east-2
test-mac-arm
ursa-i9-9960x

zagto pushed a commit to zagto/arrow that referenced this pull request Oct 7, 2022
* Update the CUDA runtime version as CUDA 9.1 images are not available anymore
* Fix passing child command arguments to "docker run"

Checked locally under a Ubuntu 20.04 host with:
```
UBUNTU=18.04 archery --debug docker run ubuntu-cuda-cpp
UBUNTU=20.04 archery --debug docker run ubuntu-cuda-cpp
```

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants