Skip to content

Switch to go-winres to create Windows resources#43431

Merged
thaJeztah merged 3 commits intomoby:masterfrom
crazy-max:goversioninfo
Apr 20, 2022
Merged

Switch to go-winres to create Windows resources#43431
thaJeztah merged 3 commits intomoby:masterfrom
crazy-max:goversioninfo

Conversation

@crazy-max
Copy link
Copy Markdown
Member

@crazy-max crazy-max commented Mar 26, 2022

follow-up docker/cli#3310 and #42872

- What I did

Replace windres with native Go implementation to be able to create a syso file which contains Microsoft Windows Version Information, an icon and the Windows Event Logging Manifest using go-winres.

- How I did it

Remove windres references and replace with go-winres. Also use go:generate for better integration and add legal copyright field.

Also add cross target to the GHA ci workflow.

- How to verify it

docker buildx bake --set *.args.DOCKER_CROSSPLATFORMS=windows/amd64 cross

@thaJeztah
Copy link
Copy Markdown
Member

Thank you!!!

@tianon @TBBle @corhere FYI; you seem like candidates that are interested in (reviewing) this 😅 🤗

@crazy-max
Copy link
Copy Markdown
Member Author

Need more work on windows build side: https://ci-next.docker.com/public/blue/organizations/jenkins/moby/detail/PR-43431/1/pipeline/282#step-283-log-343

[2022-03-26T18:31:09.692Z] At D:\gopath\src\github.com\docker\docker@tmp\durable-4df80df9\powershellWrapper.ps1:3 char:1
[2022-03-26T18:31:09.692Z] + & powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -Comm ...
[2022-03-26T18:31:09.692Z] + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2022-03-26T18:31:09.692Z]     + CategoryInfo          : NotSpecified: (cmd\dockerd\bin_windows_amd64.go:String) [], RemoteException
[2022-03-26T18:31:09.692Z]     + FullyQualifiedErrorId : NativeCommandError
[2022-03-26T18:31:09.692Z]  
[2022-03-26T18:31:09.692Z] cmd\dockerd\bin_windows_amd64.go:4: running "goversioninfo": exec: "goversioninfo": executable file not found in %PATH%

But cross is ok: https://github.com/moby/moby/runs/5704937248?check_suite_focus=true

@thaJeztah thaJeztah added this to the 21.xx milestone Mar 26, 2022
@crazy-max crazy-max force-pushed the goversioninfo branch 10 times, most recently from df9425c to d6d7787 Compare March 26, 2022 21:15
@crazy-max crazy-max marked this pull request as ready for review March 26, 2022 22:31
@crazy-max crazy-max requested a review from tianon as a code owner March 26, 2022 22:31
@crazy-max crazy-max force-pushed the goversioninfo branch 2 times, most recently from ad0ab2e to 07f92f2 Compare March 26, 2022 23:33
@thaJeztah
Copy link
Copy Markdown
Member

One failure; looks related (but could be due to how the test is run);

Stacktrace
=== RUN   TestVersion
    version_test.go:19: assertion failed: version.Version is ""
--- FAIL: TestVersion (0.03s)

@crazy-max
Copy link
Copy Markdown
Member Author

One failure; looks related (but could be due to how the test is run);

Stacktrace
=== RUN   TestVersion
    version_test.go:19: assertion failed: version.Version is ""
--- FAIL: TestVersion (0.03s)

Yes version string is not propagated the same way in the powershell script. Should be ok now.

Comment thread Dockerfile Outdated
@crazy-max crazy-max force-pushed the goversioninfo branch 2 times, most recently from 0a5a835 to b428dcf Compare March 27, 2022 09:20
@crazy-max
Copy link
Copy Markdown
Member Author

@thaJeztah Looks unrelated: https://ci-next.docker.com/public/blue/organizations/jenkins/moby/detail/PR-43431/18/pipeline/288#step-289-log-1466

[2022-03-27T09:44:02.016Z] === Failed
[2022-03-27T09:44:02.016Z] === FAIL: github.com/docker/docker/libnetwork/networkdb TestNetworkDBCRUDTableEntries (8.81s)
[2022-03-27T09:44:02.016Z]     networkdb_test.go:303: Entry existence verification test failed for node1(3ed9271ef286)
[2022-03-27T09:44:02.016Z] 2022/03/27 09:42:27 Closing DB instances...

@corhere
Copy link
Copy Markdown
Contributor

corhere commented Apr 7, 2022

@crazy-max that's a genius way of figuring out the syntax to instruct go-winres to add the message table (as a custom resource, in go-winres parlance) into the resources! Note that windmc was still needed to produce the bin file: the message table extracted from dockerd.exe 20.10.14 was compiled by windmc. As go-winres does not know how to compile message tables from an editable format, windmc would still be needed in the future to recompile the message table if one ever needed to add new entries to it. That's knowledge which could easily be forgotten if the bin file is committed to the repo directly and windmc removed from the build scripts, so if we go that route we should keep event_messages.mc around and document how to recompile it.

I think instead we could use the Manifest-based provider to publish the exact same event schema available in event_messages.mc.

It doesn't look like that would help. https://docs.microsoft.com/en-us/windows/win32/wes/writing-an-instrumentation-manifest says

After writing your manifest, use the message compiler to validate the manifest and generate the resource and header files that you include in your provider.

The message compiler it is referring to is MC.exe, the Windows SDK tool which GNU binutils windmc is a reimplementation of. There's no avoiding the message compiler, it seems.

@thaJeztah
Copy link
Copy Markdown
Member

That's knowledge which could easily be forgotten if the bin file is committed to the repo directly and windmc removed from the build scripts, so if we go that route we should keep event_messages.mc around and document how to recompile it.

💯 agreed; if instructions are needed on how to create this, we should add docs for that or a script (if possible). Not too long ago, I had to do a lot of digging to figure out how some fixtures were created (what options to pass), which was a bit of fun (see #42389)) Its ok to be hacky / minimal, as long as it's easy to discover "how to do this".

@crazy-max crazy-max changed the title Switch to goversioninfo to create Windows version info Switch to go-winres to create Windows resources Apr 8, 2022
@crazy-max
Copy link
Copy Markdown
Member Author

crazy-max commented Apr 8, 2022

Switch the implementation to use go-winres which supports event logging manifests. Also added instructions in case we want to regenerate it. Note that the event_messages.bin targets amd64 arch atm so cross against windows/arm64 might import a bad message table. I will try to generate one for arm64 if necessary.

PTAL @corhere @thaJeztah

Edit: Message table entries can be generated for any windows platform. Tried to run dockerd on a windows 11 arm64 machine as a service:

$ docker buildx bake --set *.args.DOCKER_CROSSPLATFORMS=windows/arm64 cross

Events are registered:

Copy link
Copy Markdown
Contributor

@corhere corhere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The broad strokes look good to me. There is one issue with the PowerShell script which would cause problems on Windows PowerShell v5.1. And I have some suggestions for your consideration.

Comment thread cmd/docker-proxy/genwinres_windows_386.go Outdated
Comment thread hack/make/.go-autogen.ps1
# Compile the messages
windmc hack\make\.resources-windows\event_messages.mc -h autogen\winresources\tmp -r autogen\winresources\tmp
if ($LASTEXITCODE -ne 0) { Throw "Failed to compile event message resources" }
$mkwinresContents = '{
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: PowerShell has hash tables and JSON serialization built in so you don't need to build JSON by interpolating strings.

$mkwinresContents = @{
  "RT_GROUP_ICON" = @{
    "#1" = @{
      "0409": "../../winresources/docker.ico"
    }
  }
  # etc...
} | ConvertTo-Json

Comment thread hack/make/.mkwinres
Comment thread hack/make/.go-autogen.ps1 Outdated
@crazy-max crazy-max force-pushed the goversioninfo branch 3 times, most recently from 1b986b3 to 9ad94ee Compare April 12, 2022 11:16
@crazy-max
Copy link
Copy Markdown
Member Author

Issue on Jenkins looks unrelated: https://ci-next.docker.com/public/blue/organizations/jenkins/moby/detail/PR-43431/40/pipeline/279#step-280-log-1466

[2022-04-12T11:29:08.873Z] === Failed
[2022-04-12T11:29:08.873Z] === FAIL: github.com/docker/docker/libnetwork/ipam TestRequestReleaseAddressDuplicate (0.03s)
[2022-04-12T11:29:08.873Z]     allocator_test.go:1531: IP 198.168.1.215/23 was previously allocated

@crazy-max crazy-max requested a review from thaJeztah April 12, 2022 16:57
Copy link
Copy Markdown
Contributor

@corhere corhere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! :shipit:

Copy link
Copy Markdown
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@thaJeztah
Copy link
Copy Markdown
Member

Kicked CI earlier, and all green, except for one flaky test on Windows;

=== RUN   TestNetworkDBCRUDTableEntries
    networkdb_test.go:310: Entry existence verification test failed for node2(e7bb4e7b34e3)
2022/04/20 11:21:02 Closing DB instances...
--- FAIL: TestNetworkDBCRUDTableEntries (8.04s)

Let's get this one in

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants