Enable integrations API tests for Windows CI by olljanat · Pull Request #39240 · moby/moby

olljanat · 2019-05-20T20:16:03Z

- What I did
I noticed from #37715 (comment) that Windows CI currently does not run those integration API tests so I enabled them.

- How I did it
Little bit PowerShell and small tuning to tests utils which was not fully compatible with Windows.

- How to verify it
Pass CI and verify that those new tests actually run on it.

Closes #36748

codecov · 2019-05-20T20:43:05Z

Codecov Report

❗ No coverage uploaded for pull request base (master@238f8ea). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master   #39240   +/-   ##
=========================================
  Coverage          ?   37.01%           
=========================================
  Files             ?      612           
  Lines             ?    45655           
  Branches          ?        0           
=========================================
  Hits              ?    16899           
  Misses            ?    26470           
  Partials          ?     2286

GordonTheTurtle · 2019-05-20T22:54:24Z

Please sign your commits following these rules:
https://github.com/moby/moby/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:

$ git clone -b "enable-new-integration-tests-for-win" [email protected]:olljanat/moby.git somewhere
$ cd somewhere
$ git rebase -i HEAD~842354606904
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

olljanat · 2019-06-04T20:11:31Z

RS1 will fail to that know issue but you can see from log that both RS1 and RS5 did run those integration API tests correctly:

23:04:01 INFO: Integration API tests being run from the host:
23:04:01 INFO: make.ps1 starting at 06/04/2019 20:04:01
23:04:02 Building test suite binary .\integration\build
23:04:17 Building test suite binary .\integration\config
23:04:20 Building test suite binary .\integration\container
23:04:22 Building test suite binary .\integration\image
23:04:24 Building test suite binary .\integration\network
23:04:26 Building test suite binary .\integration\plugin
23:04:27 Building test suite binary .\integration\secret
23:04:29 Building test suite binary .\integration\service
23:04:35 Building test suite binary .\integration\session
23:04:37 Building test suite binary .\integration\system
23:04:39 Building test suite binary .\integration\volume
23:04:41 Building test suite binary .\integration\network\ipvlan
23:04:43 Building test suite binary .\integration\network\macvlan
23:04:45 Building test suite binary .\integration\plugin\authz
23:04:46 Building test suite binary .\integration\plugin\common
23:04:48 Building test suite binary .\integration\plugin\graphdriver
23:04:50 Building test suite binary .\integration\plugin\logging
23:04:52 Building test suite binary .\integration\plugin\volumes
23:04:55 Building test suite binary .\integration\plugin\logging\cmd
23:04:55 Building test suite binary .\integration\plugin\logging\cmd\close_on_start
23:04:57 Building test suite binary .\integration\plugin\logging\cmd\dummy
23:04:58 Building test suite binary .\integration\plugin\volumes\cmd
23:04:58 Building test suite binary .\integration\plugin\volumes\cmd\dummy
23:05:00 Running C:\gopath\src\github.com\docker\docker\integration\build
23:05:00 INFO: Windows Base image is  microsoft/windowsservercore
23:05:00 INFO: Testing against a local daemon
23:05:02 PASS
23:05:02 Running C:\gopath\src\github.com\docker\docker\integration\config
23:05:03 INFO: Windows Base image is  microsoft/windowsservercore
23:05:03 INFO: Testing against a local daemon
23:05:03 PASS
23:05:03 Running C:\gopath\src\github.com\docker\docker\integration\container
23:05:03 INFO: Windows Base image is  microsoft/windowsservercore
23:05:03 INFO: Testing against a local daemon
23:05:57 PASS
23:05:57 Running C:\gopath\src\github.com\docker\docker\integration\image
23:05:58 INFO: Windows Base image is  microsoft/windowsservercore
23:05:58 INFO: Testing against a local daemon
23:05:58 PASS
23:05:58 Running C:\gopath\src\github.com\docker\docker\integration\network
23:05:58 INFO: Windows Base image is  microsoft/windowsservercore
23:05:58 INFO: Testing against a local daemon
23:05:58 PASS
23:05:58 Running C:\gopath\src\github.com\docker\docker\integration\plugin
23:05:58 PASS
23:05:58 Running C:\gopath\src\github.com\docker\docker\integration\secret
23:05:59 INFO: Windows Base image is  microsoft/windowsservercore
23:05:59 INFO: Testing against a local daemon
23:05:59 PASS
23:05:59 Running C:\gopath\src\github.com\docker\docker\integration\service
23:05:59 INFO: Windows Base image is  microsoft/windowsservercore
23:05:59 INFO: Testing against a local daemon
23:05:59 PASS
23:05:59 Running C:\gopath\src\github.com\docker\docker\integration\session
23:05:59 INFO: Windows Base image is  microsoft/windowsservercore
23:05:59 INFO: Testing against a local daemon
23:05:59 PASS
23:05:59 Running C:\gopath\src\github.com\docker\docker\integration\system
23:06:00 INFO: Windows Base image is  microsoft/windowsservercore
23:06:00 INFO: Testing against a local daemon
23:06:01 PASS
23:06:01 Running C:\gopath\src\github.com\docker\docker\integration\volume
23:06:01 INFO: Windows Base image is  microsoft/windowsservercore
23:06:01 INFO: Testing against a local daemon
23:06:01 PASS
23:06:01 Running C:\gopath\src\github.com\docker\docker\integration\network\ipvlan
23:06:02 INFO: Windows Base image is  microsoft/windowsservercore
23:06:02 INFO: Testing against a local daemon
23:06:02 PASS
23:06:02 Running C:\gopath\src\github.com\docker\docker\integration\network\macvlan
23:06:02 INFO: Windows Base image is  microsoft/windowsservercore
23:06:02 INFO: Testing against a local daemon
23:06:02 PASS
23:06:02 Running C:\gopath\src\github.com\docker\docker\integration\plugin\authz
23:06:02 PASS
23:06:02 Running C:\gopath\src\github.com\docker\docker\integration\plugin\common
23:06:02 INFO: Windows Base image is  microsoft/windowsservercore
23:06:02 INFO: Testing against a local daemon
23:06:02 PASS
23:06:02 Running C:\gopath\src\github.com\docker\docker\integration\plugin\graphdriver
23:06:03 INFO: Windows Base image is  microsoft/windowsservercore
23:06:03 INFO: Testing against a local daemon
23:06:03 PASS
23:06:03 Running C:\gopath\src\github.com\docker\docker\integration\plugin\logging
23:06:03 INFO: Windows Base image is  microsoft/windowsservercore
23:06:03 INFO: Testing against a local daemon
23:06:03 PASS
23:06:03 Running C:\gopath\src\github.com\docker\docker\integration\plugin\volumes
23:06:04 INFO: Windows Base image is  microsoft/windowsservercore
23:06:04 Running C:\gopath\src\github.com\docker\docker\integration\plugin\logging\cmd
23:06:04 PASS
23:06:04 Running C:\gopath\src\github.com\docker\docker\integration\plugin\logging\cmd\close_on_start
23:06:04 PASS
23:06:04 Running C:\gopath\src\github.com\docker\docker\integration\plugin\logging\cmd\dummy
23:06:04 PASS
23:06:04 Running C:\gopath\src\github.com\docker\docker\integration\plugin\volumes\cmd
23:06:04 PASS
23:06:04 Running C:\gopath\src\github.com\docker\docker\integration\plugin\volumes\cmd\dummy
23:06:04 PASS

integration/volume/volume_test.go

thaJeztah · 2019-06-05T11:32:06Z

❤️ With that RS1 fix merged; could you rebase to re-trigger CI and re-run with the fix in place?

thaJeztah · 2019-06-05T11:34:01Z

/cc @ddebroy @StefanScherer @jhowardmsft ptal

thaJeztah · 2019-06-05T14:22:58Z

I can give it a quick go

thaJeztah · 2019-06-05T15:04:40Z

Opened #39325, but I see it failing on RS1 as well, so there might be an actual issue somewhere 🤔

olljanat · 2019-06-05T16:36:59Z

Included #39325 here and looks that it fixes CI issue. Will rebase after that one is merged.

StefanScherer

SGTM

thaJeztah · 2019-06-06T18:22:25Z

#39325 was merged, so you can rebase if you want

olljanat · 2019-06-06T19:33:42Z

Done and now both RS1 and RS5 passed CI 💚

thaJeztah · 2019-06-06T19:48:58Z

Are any of the tests actually run? Looking at the duration of running that test (less than a minute) 🤔 (possibly most are non-windows, but curious)

olljanat · 2019-06-06T21:12:35Z

Are any of the tests actually run? Looking at the duration of running that test (less than a minute) (possibly most are non-windows, but curious)

It does test something as tests was failing to error that there is no command top and test TestVolumesCreateAndList failed to error that cannot find container with correct name without those modifications which I included to this.

However some flag is most probably missing as it does not print info that which tests it actually runs like Linux does (even when I run it locally).

thaJeztah · 2019-06-07T11:30:12Z

However some flag is most probably missing as it does not print info that which tests it actually runs like Linux does (even when I run it locally).

Hm, right. @vdemeester do you know from the top of your head?

Signed-off-by: Olli Janatuinen <[email protected]>

olljanat · 2019-06-10T18:48:04Z

Included verbose flag -test.vnow it also tell which tests it run 😎

thaJeztah

LGTM 🎉

thaJeztah · 2019-06-11T18:36:15Z

ping @cpuguy83 @jhowardmsft LGTY?

lowenna · 2019-06-11T18:44:05Z

LGTM

olljanat requested a review from tianon as a code owner May 20, 2019 20:16

GordonTheTurtle added the status/0-triage label May 20, 2019

olljanat force-pushed the enable-new-integration-tests-for-win branch from 7c41f55 to d5d4629 Compare May 20, 2019 22:54

GordonTheTurtle added the dco/no Automatically set by a bot when one of the commits lacks proper signature label May 20, 2019

olljanat force-pushed the enable-new-integration-tests-for-win branch from d5d4629 to 78c046c Compare May 20, 2019 22:55

GordonTheTurtle removed the dco/no Automatically set by a bot when one of the commits lacks proper signature label May 20, 2019

olljanat force-pushed the enable-new-integration-tests-for-win branch from 78c046c to bffc2c3 Compare May 20, 2019 23:16

olljanat mentioned this pull request May 22, 2019

CI: Display error details on Windows unit and integration tests and clean Go cache before build #39232

Closed

olljanat force-pushed the enable-new-integration-tests-for-win branch 2 times, most recently from 82a9dd1 to 1828d5f Compare May 22, 2019 14:32

GordonTheTurtle assigned MHBauer Jun 4, 2019

olljanat force-pushed the enable-new-integration-tests-for-win branch 7 times, most recently from b2dfa5e to 6eb1c8a Compare June 4, 2019 19:52

olljanat changed the title ~~WIP: Enable integrations API tests for Windows CI~~ Enable integrations API tests for Windows CI Jun 4, 2019

thaJeztah reviewed Jun 5, 2019

View reviewed changes

integration/volume/volume_test.go Outdated Show resolved Hide resolved

thaJeztah added area/testing platform/windows process/cherry-pick status/2-code-review and removed status/0-triage labels Jun 5, 2019

thaJeztah mentioned this pull request Jun 5, 2019

Harden TestPsListContainersFilterExited #39325

Merged

olljanat force-pushed the enable-new-integration-tests-for-win branch from 021309e to 36340fc Compare June 5, 2019 15:46

StefanScherer approved these changes Jun 5, 2019

View reviewed changes

olljanat force-pushed the enable-new-integration-tests-for-win branch from 36340fc to 3d40f65 Compare June 6, 2019 18:29

thaJeztah mentioned this pull request Jun 7, 2019

[18.09 backport] Harden TestPsListContainersFilterExited docker-archive/engine#269

Merged

Enable integrations API tests for Windows CI

2f22247

Signed-off-by: Olli Janatuinen <[email protected]>

olljanat force-pushed the enable-new-integration-tests-for-win branch from 3d40f65 to 2f22247 Compare June 10, 2019 18:33

thaJeztah approved these changes Jun 11, 2019

View reviewed changes

thaJeztah added status/4-merge and removed status/2-code-review labels Jun 11, 2019

olljanat mentioned this pull request Jun 11, 2019

Windows: TerminateOnLastHandleClosed #39354

Closed

cpuguy83 merged commit 02dceea into moby:master Jun 11, 2019

This was referenced Jun 12, 2019

[18.09 backport] Enable integrations API tests for Windows CI docker-archive/engine#276

Merged

[19.03 backport] Enable integrations API tests for Windows CI docker-archive/engine#277

Merged

thaJeztah added process/cherry-picked and removed process/cherry-pick labels Jun 12, 2019

olljanat mentioned this pull request Aug 1, 2019

Windows "integration" tests report incorrect status #39576

Closed

thaJeztah added this to the 20.03.0 milestone Apr 2, 2020

olljanat mentioned this pull request Sep 19, 2020

Windows CI: Enable more integration tests #38469

Merged

Conversation

olljanat commented May 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented May 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

GordonTheTurtle commented May 20, 2019

Uh oh!

olljanat commented Jun 4, 2019

Uh oh!

Uh oh!

thaJeztah commented Jun 5, 2019

Uh oh!

thaJeztah commented Jun 5, 2019

Uh oh!

thaJeztah commented Jun 5, 2019

Uh oh!

thaJeztah commented Jun 5, 2019

Uh oh!

olljanat commented Jun 5, 2019

Uh oh!

StefanScherer left a comment

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Jun 6, 2019

Uh oh!

olljanat commented Jun 6, 2019

Uh oh!

thaJeztah commented Jun 6, 2019

Uh oh!

olljanat commented Jun 6, 2019

Uh oh!

thaJeztah commented Jun 7, 2019

Uh oh!

olljanat commented Jun 10, 2019

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Jun 11, 2019

Uh oh!

lowenna commented Jun 11, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

olljanat commented May 20, 2019 •

edited

Loading

codecov bot commented May 20, 2019 •

edited

Loading