CI infrastructure doesn't validate Hyper-V Isolation for Windows containers #43395
Description
Description
While extending device support for Windows Containers in #43368, I noticed that the Hyper-V-based tests failed for what looked like systematic reasons (i.e. not because of my code) in some or all runs.
Looking further, I realised there are no integration tests run with Hyper-V isolation, except when Hyper-V isolation is the daemon default, i.e. on Windows Client, which is not tested by CI.
Steps to reproduce the issue:
- Merge Introduce
://syntax for Windows Devices in DeviceMapping.PathOnHost #43368 or branch from it if it's not yet merged. - Remove the skip with description "FIXME. HyperV isolation setup is probably incorrect in the test in integration/container/devices_windows_test.go.
- Create a PR from that to get a CI run
Describe the results you received:
client.ContainerStart failed with issues unrelated to the device spec.
Windows Server 2022 parallel:
hcs::CreateComputeSystem : The request is not supported.
Windows Server 2022 w/Containerd parallel:
Error response from daemon: failed to create shim: error in preparing config doc: failed to get host processor information: failed to retrieve processor and processor topology information: hcs::GetServiceProperties: Catastrophic failure: unknown
Describe the results you expected:
client.ContainerStart succeeds, or fails due to an unusable Device spec.
Additional information you deem important (e.g. issue happens only occasionally):
I saw at least one success for these tests with Hyper-V on the Windows RS5 parallel.
My guess of the failure cause was that Hyper-V isolation in a VM requires Nested Virtualisation support, and that it's enabled for the Windows Server 2019 build node but not the Windows Server 2022 build node.
I'd suggest adding explicit integration tests for Hyper-V isolation on Windows Server, and perhaps skipping them if run an environment that does not support nested virtualisation.
Output of docker version:
Note: this was from the win-2022 parallel for the current run of #43368's "Run tests" stage, not the "Print info" stage.
[2022-03-19T04:25:08.311Z] INFO: Docker version of the daemon under test
[2022-03-19T04:25:08.311Z]
[2022-03-19T04:25:08.311Z] Client:
[2022-03-19T04:25:08.311Z] Version: 17.06.2-ce
[2022-03-19T04:25:08.311Z] API version: 1.30
[2022-03-19T04:25:08.311Z] Go version: go1.8.3
[2022-03-19T04:25:08.311Z] Git commit: cec0b72
[2022-03-19T04:25:08.311Z] Built: Tue Sep 5 19:57:19 2017
[2022-03-19T04:25:08.311Z] OS/Arch: windows/amd64
[2022-03-19T04:25:08.311Z]
[2022-03-19T04:25:08.311Z] Server:
[2022-03-19T04:25:08.311Z] Version: 0.0.0-dev
[2022-03-19T04:25:08.311Z] API version: 1.42 (minimum version 1.24)
[2022-03-19T04:25:08.311Z] Go version: go1.18
[2022-03-19T04:25:08.311Z] Git commit: 6185cdf174
[2022-03-19T04:25:08.311Z] Built: 03/19/2022 04:22:55
[2022-03-19T04:25:08.311Z] OS/Arch: windows/amd64
[2022-03-19T04:25:08.311Z] Experimental: false
Output of docker info:
Note: this was from the win-2022 parallel for the current run of #43368's "Run tests" stage, not the "Print info" stage.
[2022-03-19T04:25:08.311Z] INFO: Docker info of the daemon under test
[2022-03-19T04:25:08.311Z]
[2022-03-19T04:25:08.311Z] Containers: 0
[2022-03-19T04:25:08.311Z] Running: 0
[2022-03-19T04:25:08.311Z] Paused: 0
[2022-03-19T04:25:08.311Z] Stopped: 0
[2022-03-19T04:25:08.311Z] Images: 0
[2022-03-19T04:25:08.311Z] Server Version: 0.0.0-dev
[2022-03-19T04:25:08.311Z] Storage Driver: windowsfilter
[2022-03-19T04:25:08.311Z] Windows:
[2022-03-19T04:25:08.312Z] Logging Driver: json-file
[2022-03-19T04:25:08.312Z] Plugins:
[2022-03-19T04:25:08.312Z] Volume: local
[2022-03-19T04:25:08.312Z] Network: ics internal l2bridge l2tunnel nat null overlay private transparent
[2022-03-19T04:25:08.312Z] Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
[2022-03-19T04:25:08.312Z] Swarm: inactive
[2022-03-19T04:25:08.312Z] Default Isolation: process
[2022-03-19T04:25:08.312Z] Kernel Version: 10.0 20348 (20348.1.amd64fre.fe_release.210507-1500)
[2022-03-19T04:25:08.312Z] Operating System: Microsoft Windows Server Version 21H2 (OS Build 20348.230)
[2022-03-19T04:25:08.312Z] OSType: windows
[2022-03-19T04:25:08.312Z] Architecture: x86_64
[2022-03-19T04:25:08.312Z] CPUs: 4
[2022-03-19T04:25:08.312Z] Total Memory: 32GiB
[2022-03-19T04:25:08.312Z] Name: azwin-2-442a71
[2022-03-19T04:25:08.312Z] ID: UMFK:HN3Z:3TSI:UXJ7:P462:2J3M:6Q5I:TE46:7LMZ:GL3B:WPWN:7WJY
[2022-03-19T04:25:08.312Z] Docker Root Dir: D:\CI\PR-43368\27\daemon
[2022-03-19T04:25:08.312Z] Debug Mode (client): false
[2022-03-19T04:25:08.312Z] Debug Mode (server): true
[2022-03-19T04:25:08.312Z] File Descriptors: -1
[2022-03-19T04:25:08.312Z] Goroutines: 16
[2022-03-19T04:25:08.312Z] System Time: 2022-03-19T04:25:05.3960347Z
[2022-03-19T04:25:08.312Z] EventsListeners: 0
[2022-03-19T04:25:08.312Z] Registry: https://index.docker.io/v1/
[2022-03-19T04:25:08.312Z] Labels:
[2022-03-19T04:25:08.312Z] Experimental: false
[2022-03-19T04:25:08.312Z] Insecure Registries:
[2022-03-19T04:25:08.312Z] 127.0.0.0/8
[2022-03-19T04:25:08.312Z] Live Restore Enabled: false
Additional environment details (AWS, VirtualBox, physical, etc.):
This is all on the CI infrastructure, which is AFAIK Azure-hosted VMs.
I think that the following check in PowerShell would be enough to confirm that Hyper-V Isolation is possible, if Process Isolation requirements are otherwise met:
(Get-CimInstance -ClassName Win32_ComputerSystem).HypervisorPresent
I have not checked that though, as I don't currently have any machines around where Hyper-V Isolation doesn't work.