ci: Run Windows native task on GitHub Actions #28173

hebasto · 2023-07-27T21:10:23Z

Thus, someone would have to sponsor an amount of roughly 5kUSD/mo for those two tasks.

If the goal is to stay on a free plan, I think the only option is GitHub Actions CI.

Historical context:

Security concerns:

GITHUB_TOKEN permissions (from the build log in my personal repo):

2023-07-27T07:30:17.8313534Z ##[group]GITHUB_TOKEN Permissions
2023-07-27T07:30:17.8314113Z Contents: read
2023-07-27T07:30:17.8314608Z Metadata: read
2023-07-27T07:30:17.8314957Z Packages: read
2023-07-27T07:30:17.8315233Z ##[endgroup]

Comparison of resources:

Resource	Current, Cirrus CI	Suggested, GitHub Actions
CPU	6	2
RAM, GB	12	7

The TEST_RUNNER_TIMEOUT_FACTOR variable is set to the current default value for all CI tasks:

bitcoin/ci/test/00_setup_env.sh

Line 48 in 64440bb

export TEST_RUNNER_TIMEOUT_FACTOR=${TEST_RUNNER_TIMEOUT_FACTOR:-40}

DrahtBot · 2023-07-27T21:10:26Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	real-or-random
Stale ACK	dergoegge

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

No conflicts as of last run.

maflcko · 2023-07-28T05:46:38Z

Can you remove the Windows Cirrus task here, as it will need to be removed either way? Also, a link to a build would be nice, before this is enabled here.

maflcko · 2023-07-28T05:47:46Z

Maybe also print the GITHUB_TOKEN permissions at every start of the task, just to double check they are read-only?

.github/workflows/ci.yml

hebasto · 2023-07-28T10:51:27Z

Can you remove the Windows Cirrus task here, as it will need to be removed either way?

Done.

Maybe also print the GITHUB_TOKEN permissions at every start of the task, just to double check they are read-only?

They are printed by default. To see them, in https://github.com/hebasto/bitcoin/actions/runs/5685393993/job/15410187833, one needs to expand "Set up job", then expand "GIHUB_TOKEN Permissions".

maflcko · 2023-07-28T10:55:49Z

Ok, so I guess someone needs to enable Actions in both orgs for this to proceed.

maflcko · 2023-07-28T10:56:48Z

In the meantime it may be good to do 10 runs and then check how many of them fail

hebasto · 2023-07-29T09:10:47Z

In the meantime it may be good to do 10 runs and then check how many of them fail

Observing intermittent "Error: no RPC connection". Converting this PR to a draft for now.

UPD. Timeout has been adjusted.

hebasto · 2023-07-30T14:35:22Z

In the meantime it may be good to do 10 runs and then check how many of them fail

10 runs are green for the recent commit:

Ok, so I guess someone needs to enable Actions in both orgs for this to proceed.

@sipa @fanquake

Mind enabling GitHub Actions in the https://github.com/bitcoin/bitcoin repository?

I'm suggesting the repository "Actions permissions" as follows:

hebasto · 2023-08-01T09:00:51Z

Rebased on top of the merged #28188.

hebasto · 2023-08-05T07:52:00Z

Converting this PR to a draft as it conflicts with #28187.

Please review #28187 first.

hebasto · 2023-08-19T17:19:52Z

#28292 is a blocker for this PR.

maflcko · 2023-08-21T08:31:43Z

#28292 is a blocker for this PR.

Is it? It should only affect the cache hit percentage, no? And the cache hit percentage shouldn't affect the behavior of this pull, except for the cache hit percentage, potentially.

hebasto · 2023-08-21T10:42:05Z

#28292 is a blocker for this PR.

Is it? It should only affect the cache hit percentage, no?

Yes, that is what I mean. Without #28292, caches for the master branch are to be evicted very often. And this PR makes it roughly 2 times more often.

maflcko · 2023-08-21T10:51:13Z

Ok, fair enough. But that shouldn't be a blocker to rebase this, and for code review to happen on the rebase.

dergoegge · 2023-08-24T10:05:53Z

By the way, what's an "lgtm ACK"?

(at least for me) it's just a normal ack pre-fixed with "looks good to me"

dergoegge

reACK a3ebaa2

hebasto · 2023-08-25T08:25:38Z

Friendly ping @MarcoFalke :)

.github/workflows/ci.yml

hebasto · 2023-08-25T12:54:40Z

Updated a3ebaa2 -> 6a43372 (pr28173.11 -> pr28173.12):

rebased
addressed @MarcoFalke's comment

maflcko · 2023-08-25T13:03:50Z

lgtm

maflcko · 2023-08-28T08:13:00Z

Would be good to merge it, otherwise the CI will break this week.

maflcko · 2023-08-28T18:42:43Z

.github/workflows/ci.yml

+            C:\ProgramData\chocolatey\lib\ccache
+            C:\ProgramData\chocolatey\bin\ccache.exe
+            C:\ccache\cl.exe
+          key: ${{ github.job }}-ccache-installation-${{ env.CI_CCACHE_VERSION }}


Looks like the cache key will additionally include the branch name / context, so this will cycle the cache. See https://github.com/bitcoin/bitcoin/actions/caches

win64-native-ccache-installation-4.7.5 2.7 MB cached August 28, 2023 08:55 refs/pull/27601/merge win64-native-ccache-installation-4.7.5 2.7 MB cached August 28, 2023 07:43 refs/pull/26312/merge

#28173 (comment):

While key remains the same, pull requests will restore cache from the master branch and make no new saves.

A few pull requests created their caches since they had not been created for the master branch yet.

All recent pull requests skipped saving their caches because they hit the caches from the master branch.

maflcko · 2023-08-28T18:43:27Z

.github/workflows/ci.yml

+        uses: actions/cache@v3
+        with:
+          path: C:/vcpkg/downloads/tools
+          key: ${{ github.job }}-vcpkg-tools


same, etc ...

#28173 (comment)

…nal tests for pull requests 62ab3e9 ci, windows: Do not run extended functional tests for pull requests (Hennadii Stepanov) Pull request description: This PR is intended to speed up the CI feedback for pull requests: - a [PR](https://github.com/bitcoin/bitcoin/actions/runs/6019964104?pr=28196) opened against the current master branch: ![image](https://github.com/bitcoin/bitcoin/assets/32963518/481a70eb-13f3-40c9-8f6a-ca2f06350158) - this PR: ![image](https://github.com/bitcoin/bitcoin/assets/32963518/2582307f-7b72-4816-b5be-e84d5e4a3016) Suggested in bitcoin/bitcoin#28173 (comment): > An alternative would be to run them on non-pr pushes only. Failures should be rare enough to deal with them post-merge. ACKs for top commit: MarcoFalke: lgtm ACK 62ab3e9 if https://github.com/hebasto/bitcoin/actions/runs/6023862001/job/16341417883 is green Tree-SHA512: e937efc5c03290f5d246ce1b0638dc72f39ef1d509ba5d2063f92bfe9157c602e6a952a9558dfc6413bbc5209fe55280b3278a0e4079773b8cc9ff236c8f9246

maflcko · 2023-09-01T07:12:10Z

.cirrus.yml

-    # Increase the dynamic port range to the maximum allowed value to mitigate "OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted".
-    # See: https://learn.microsoft.com/en-us/biztalk/technical-guides/settings-that-can-be-modified-to-improve-network-performance
-    - netsh int ipv4 set dynamicport tcp start=1025 num=64511
-    - netsh int ipv6 set dynamicport tcp start=1025 num=64511


Is this no longer needed? Looks like the functional tests fail on almost every push?

https://github.com/bitcoin/bitcoin/actions/runs/6042244259/job/16396918299?pr=26697

https://github.com/bitcoin/bitcoin/actions/runs/6045738603/job/16406226269?pr=27866

No, logs say nothing about WinError 10048 error.

See #28384 for a proposed fix.

…s for pull requests 62ab3e9 ci, windows: Do not run extended functional tests for pull requests (Hennadii Stepanov) Pull request description: This PR is intended to speed up the CI feedback for pull requests: - a [PR](https://github.com/bitcoin/bitcoin/actions/runs/6019964104?pr=28196) opened against the current master branch: ![image](https://github.com/bitcoin/bitcoin/assets/32963518/481a70eb-13f3-40c9-8f6a-ca2f06350158) - this PR: ![image](https://github.com/bitcoin/bitcoin/assets/32963518/2582307f-7b72-4816-b5be-e84d5e4a3016) Suggested in bitcoin#28173 (comment): > An alternative would be to run them on non-pr pushes only. Failures should be rare enough to deal with them post-merge. ACKs for top commit: MarcoFalke: lgtm ACK 62ab3e9 if https://github.com/hebasto/bitcoin/actions/runs/6023862001/job/16341417883 is green Tree-SHA512: e937efc5c03290f5d246ce1b0638dc72f39ef1d509ba5d2063f92bfe9157c602e6a952a9558dfc6413bbc5209fe55280b3278a0e4079773b8cc9ff236c8f9246

DrahtBot added the Tests label Jul 27, 2023

maflcko reviewed Jul 28, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

hebasto force-pushed the 230727-actions branch from a22565d to 315bb38 Compare July 28, 2023 10:45

hebasto marked this pull request as draft July 29, 2023 09:11

hebasto force-pushed the 230727-actions branch from 315bb38 to efd835c Compare July 29, 2023 17:39

hebasto marked this pull request as ready for review July 30, 2023 14:22

hebasto mentioned this pull request Jul 30, 2023

ci: Future of macOS and Windows MSVC CI tasks #28098

Closed

This was referenced Jul 31, 2023

ci: Use documented CCACHE_MAXSIZE instead of CCACHE_SIZE #28188

Merged

ci: Run "macOS native x86_64" job on GitHub Actions #28187

Merged

DrahtBot added the Needs rebase label Jul 31, 2023

hebasto force-pushed the 230727-actions branch from efd835c to b3d6338 Compare August 1, 2023 08:59

DrahtBot removed the Needs rebase label Aug 1, 2023

hebasto marked this pull request as draft August 5, 2023 07:52

DrahtBot mentioned this pull request Aug 14, 2023

ci: Run "macOS 11.0 [gui, no tests] [jammy]" job on GitHub Actions #28265

Closed

DrahtBot added the Needs rebase label Aug 15, 2023

hebasto force-pushed the 230727-actions branch from b3d6338 to d0e9f5a Compare August 21, 2023 11:05

DrahtBot added CI failed and removed CI failed labels Aug 24, 2023

hebasto requested review from dergoegge and maflcko August 24, 2023 09:29

dergoegge approved these changes Aug 24, 2023

View reviewed changes

maflcko reviewed Aug 25, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

maflcko reviewed Aug 25, 2023

View reviewed changes

.github/workflows/ci.yml Show resolved Hide resolved

maflcko reviewed Aug 25, 2023

View reviewed changes

.github/workflows/ci.yml Show resolved Hide resolved

maflcko reviewed Aug 25, 2023

View reviewed changes

.github/workflows/ci.yml Show resolved Hide resolved

ci: Run "Win64 native" job on GitHub Actions

6a43372

hebasto force-pushed the 230727-actions branch from a3ebaa2 to 6a43372 Compare August 25, 2023 12:50

hebasto requested a review from dergoegge August 25, 2023 15:04

maflcko requested a review from fanquake August 28, 2023 08:11

maflcko added this to the 26.0 milestone Aug 28, 2023

fanquake merged commit db57574 into bitcoin:master Aug 28, 2023

maflcko reviewed Aug 28, 2023

View reviewed changes

hebasto deleted the 230727-actions branch August 28, 2023 20:21

hebasto mentioned this pull request Aug 28, 2023

ci, windows: Do not run extended functional tests for pull requests #28360

Merged

maflcko reviewed Sep 1, 2023

View reviewed changes

sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Sep 26, 2023

Merge bitcoin#28173: ci: Run Windows native task on GitHub Actions

4a7a230

hebasto mentioned this pull request Mar 8, 2024

ci: switch back to executing functional tests with powershell #29581

Closed

bitcoin locked and limited conversation to collaborators Aug 31, 2024

ci: Run Windows native task on GitHub Actions #28173

ci: Run Windows native task on GitHub Actions #28173

Uh oh!

Conversation

hebasto commented Jul 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DrahtBot commented Jul 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews

Conflicts

Uh oh!

maflcko commented Jul 28, 2023

Uh oh!

maflcko commented Jul 28, 2023

Uh oh!

Uh oh!

hebasto commented Jul 28, 2023

Uh oh!

maflcko commented Jul 28, 2023

Uh oh!

maflcko commented Jul 28, 2023

Uh oh!

hebasto commented Jul 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hebasto commented Jul 30, 2023

Uh oh!

hebasto commented Aug 1, 2023

Uh oh!

hebasto commented Aug 5, 2023

Uh oh!

hebasto commented Aug 19, 2023

Uh oh!

maflcko commented Aug 21, 2023

Uh oh!

hebasto commented Aug 21, 2023

Uh oh!

maflcko commented Aug 21, 2023

Uh oh!

dergoegge commented Aug 24, 2023

Uh oh!

dergoegge left a comment

Choose a reason for hiding this comment

Uh oh!

hebasto commented Aug 25, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hebasto commented Aug 25, 2023

Uh oh!

maflcko commented Aug 25, 2023

Uh oh!

maflcko commented Aug 28, 2023

Uh oh!

maflcko Aug 28, 2023

Choose a reason for hiding this comment

Uh oh!

hebasto Aug 28, 2023

Choose a reason for hiding this comment

Uh oh!

maflcko Aug 28, 2023

Choose a reason for hiding this comment

Uh oh!

hebasto Aug 28, 2023

Choose a reason for hiding this comment

Uh oh!

maflcko Sep 1, 2023

Choose a reason for hiding this comment

Uh oh!

hebasto Sep 1, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

hebasto commented Jul 27, 2023 •

edited

Loading

DrahtBot commented Jul 27, 2023 •

edited

Loading

hebasto commented Jul 29, 2023 •

edited

Loading