Skip to content

[release/1.7 backport] update to go1.21.3, go1.20.10#9227

Closed
thaJeztah wants to merge 10 commits intocontainerd:release/1.7from
thaJeztah:1.7_backport_update_golang_1.21.3
Closed

[release/1.7 backport] update to go1.21.3, go1.20.10#9227
thaJeztah wants to merge 10 commits intocontainerd:release/1.7from
thaJeztah:1.7_backport_update_golang_1.21.3

Conversation

@thaJeztah
Copy link
Copy Markdown
Member

@thaJeztah thaJeztah commented Oct 12, 2023

thaJeztah and others added 6 commits October 12, 2023 15:13
This reverts commit 423693d.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
> github.com/containerd/containerd/contrib/apparmor
> github.com/containerd/containerd/contrib/apparmor
> Running go-fuzz -tags gofuzz -func FuzzLoadDefaultProfile -o fuzz_FuzzLoadDefaultProfile.a github.com/containerd/containerd/contrib/apparmor
> /usr/bin/ld: /usr/bin/ld: DWARF error: invalid or unhandled FORM value: 0x25
> fuzz_FuzzLoadDefaultProfile.a(000021.o): in function `_cgo_9c8efe9babca_C2func_res_search':
> cgo_unix_cgo_res.cgo2.c:(.text+0x32): undefined reference to `__res_search'
> /usr/bin/ld: fuzz_FuzzLoadDefaultProfile.a(000021.o): in function `_cgo_9c8efe9babca_Cfunc_res_search':
> cgo_unix_cgo_res.cgo2.c:(.text+0x81): undefined reference to `__res_search'
> clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
> 2023-08-11 14:25:45,433 - root - ERROR - Building fuzzers failed.
> 2023-08-11 14:25:45,433 - root - ERROR - Error building fuzzers for (commit: 432d86b, pr_ref: refs/pull/8957/merge).

Signed-off-by: Akihiro Suda <[email protected]>
(cherry picked from commit 5d9bf7d)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Tests in pkg/cri/[sb]server/container_create_linux_test.go depends on go:noinline
since Go 1.21.

e.g.,
> ```
> === FAIL: pkg/cri/sbserver TestGenerateSeccompSecurityProfileSpecOpts/should_set_default_seccomp_when_seccomp_is_runtime/default (0.00s)
>     container_create_linux_test.go:1013:
>         	Error Trace:	/home/runner/work/containerd/containerd/pkg/cri/sbserver/container_create_linux_test.go:1013
>         	Error:      	Not equal:
>         	            	expected: 0x263d880
>         	            	actual  : 0x263cbc0
>         	Test:       	TestGenerateSeccompSecurityProfileSpecOpts/should_set_default_seccomp_when_seccomp_is_runtime/default
> ```

See comments in PR 8957.

Thanks to Wei Fu for analyzing this.

Co-authored-by: Wei Fu <[email protected]>
Signed-off-by: Akihiro Suda <[email protected]>
(cherry picked from commit 0f043ae)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The default version of MinGW and GCC on the GitHub-hosted Windows 2019
runners compile fine but lead to linker errors during runtime.

Signed-off-by: Nashwan Azhari <[email protected]>
(cherry picked from commit c883410)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Akihiro Suda <[email protected]>
(cherry picked from commit 5756cb0)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
https://go.dev/doc/go1.21

Signed-off-by: Akihiro Suda <[email protected]>
(cherry picked from commit eacd74c)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
@k8s-ci-robot
Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

fuweid and others added 4 commits October 12, 2023 17:10
Signed-off-by: Wei Fu <[email protected]>
(cherry picked from commit 1dced31)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
go1.21.1 (released 2023-09-06) includes four security fixes to the cmd/go,
crypto/tls, and html/template packages, as well as bug fixes to the compiler,
the go command, the linker, the runtime, and the context, crypto/tls,
encoding/gob, encoding/xml, go/types, net/http, os, and path/filepath packages.
See the Go 1.21.1 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.21.1+label%3ACherryPickApproved

full diff: golang/go@go1.21.0...go1.21.1

From the security mailing:

[security] Go 1.21.1 and Go 1.20.8 are released

Hello gophers,

We have just released Go versions 1.21.1 and 1.20.8, minor point releases.

These minor releases include 4 security fixes following the security policy:

- cmd/go: go.mod toolchain directive allows arbitrary execution
  The go.mod toolchain directive, introduced in Go 1.21, could be leveraged to
  execute scripts and binaries relative to the root of the module when the "go"
  command was executed within the module. This applies to modules downloaded using
  the "go" command from the module proxy, as well as modules downloaded directly
  using VCS software.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-39320 and Go issue https://go.dev/issue/62198.

- html/template: improper handling of HTML-like comments within script contexts
  The html/template package did not properly handle HMTL-like "<!--" and "-->"
  comment tokens, nor hashbang "#!" comment tokens, in <script> contexts. This may
  cause the template parser to improperly interpret the contents of <script>
  contexts, causing actions to be improperly escaped. This could be leveraged to
  perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39318 and Go issue https://go.dev/issue/62196.

- html/template: improper handling of special tags within script contexts
  The html/template package did not apply the proper rules for handling occurrences
  of "<script", "<!--", and "</script" within JS literals in <script> contexts.
  This may cause the template parser to improperly consider script contexts to be
  terminated early, causing actions to be improperly escaped. This could be
  leveraged to perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39319 and Go issue https://go.dev/issue/62197.

- crypto/tls: panic when processing post-handshake message on QUIC connections
  Processing an incomplete post-handshake message for a QUIC connection caused a panic.

  Thanks to Marten Seemann for reporting this issue.

  This is CVE-2023-39321 and CVE-2023-39322 and Go issue https://go.dev/issue/62266.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit eaf6071)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
go1.21.2 (released 2023-10-05) includes one security fixes to the cmd/go package,
as well as bug fixes to the compiler, the go command, the linker, the runtime,
and the runtime/metrics package. See the Go 1.21.2 milestone on our issue
tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.21.2+label%3ACherryPickApproved

full diff: golang/go@go1.21.1...go1.21.2

From the security mailing:

[security] Go 1.21.2 and Go 1.20.9 are released

Hello gophers,

We have just released Go versions 1.21.2 and 1.20.9, minor point releases.

These minor releases include 1 security fixes following the security policy:

- cmd/go: line directives allows arbitrary execution during build

  "//line" directives can be used to bypass the restrictions on "//go:cgo_"
  directives, allowing blocked linker and compiler flags to be passed during
  compliation. This can result in unexpected execution of arbitrary code when
  running "go build". The line directive requires the absolute path of the file in
  which the directive lives, which makes exploting this issue significantly more
  complex.

  This is CVE-2023-39323 and Go issue https://go.dev/issue/63211.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 6b752b5)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
go1.21.3 (released 2023-10-10) includes a security fix to the net/http package.
See the Go 1.21.3 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.21.3+label%3ACherryPickApproved

full diff: golang/go@go1.21.2...go1.21.3

From the security mailing:

[security] Go 1.21.3 and Go 1.20.10 are released

Hello gophers,

We have just released Go versions 1.21.3 and 1.20.10, minor point releases.

These minor releases include 1 security fixes following the security policy:

- net/http: rapid stream resets can cause excessive work

  A malicious HTTP/2 client which rapidly creates requests and
  immediately resets them can cause excessive server resource consumption.
  While the total number of requests is bounded to the
  http2.Server.MaxConcurrentStreams setting, resetting an in-progress
  request allows the attacker to create a new request while the existing
  one is still executing.

  HTTP/2 servers now bound the number of simultaneously executing
  handler goroutines to the stream concurrency limit. New requests
  arriving when at the limit (which can only happen after the client
  has reset an existing, in-flight request) will be queued until a
  handler exits. If the request queue grows too large, the server
  will terminate the connection.

  This issue is also fixed in golang.org/x/net/http2 v0.17.0,
  for users manually configuring HTTP/2.

  The default stream concurrency limit is 250 streams (requests)
  per HTTP/2 connection. This value may be adjusted using the
  golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
  setting and the ConfigureServer function.

  This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
  This is also tracked by CVE-2023-44487.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 313f514)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
@thaJeztah thaJeztah force-pushed the 1.7_backport_update_golang_1.21.3 branch from 9ce98eb to 0d4edcd Compare October 12, 2023 15:10
@thaJeztah thaJeztah marked this pull request as ready for review October 12, 2023 17:55
@thaJeztah
Copy link
Copy Markdown
Member Author

Failures are due to the CRIU ppa failing to be installed; did we have a fix for that somewhere, or is it just broken?

@estesp
Copy link
Copy Markdown
Member

estesp commented Oct 12, 2023

Looks like after Thu, 12 Oct 2023 15:22:45 GMT (+/- 10 seconds) .. any "install criu via PPA" action has failed. Both PRs for updating Go have a few jobs that pulled from the PPA just before that time and each had 2 runs that pulled just after that threshold and failed. All retries are now failing so looks like something is down with the criu PPA or the mirror used in GHA.

@thaJeztah
Copy link
Copy Markdown
Member Author

Oh! Interesting; first run failed to add the PPA

 sudo add-apt-repository ppa:criu/ppa
  sudo apt-get update
  sudo apt-get install -y criu
  shell: /usr/bin/bash -e {0}
  env:
    GO_VERSION: 1.21.3
    GOTEST: gotestsum --
Cannot add PPA: 'ppa:~criu/ubuntu/ppa'.
The team named '~criu' has no PPA named 'ubuntu/ppa'
Please choose from the following available PPAs:
 * 'ppa':  CRIU PPA
Error: Process completed with exit code 1.

Second run shows a python failure? Is this because add-apt-repository is deprecated?

sudo add-apt-repository ppa:criu/ppa
  sudo apt-get update
  sudo apt-get install -y criu
  shell: /usr/bin/bash -e {0}
  env:
    GO_VERSION: 1.21.3
    GOTEST: gotestsum --
Traceback (most recent call last):
  File "/usr/bin/add-apt-repository", line 137, in <module>
    shortcut = shortcut_handler(line)
  File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 885, in shortcut_handler
    ret = factory(shortcut)
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 469, in shortcut_handler
    return PPAShortcutHandler(shortcut)
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 426, in __init__
    info = get_ppa_info(self.shortcut)
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 392, in get_ppa_info
    _get_suggested_ppa_message(user, ppa))
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 349, in _get_suggested_ppa_message
    lp_user = get_info_from_lp(LAUNCHPAD_USER_API % user)
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 104, in get_info_from_lp
    return get_info_from_https(lp_url, True)
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 98, in get_info_from_https
    return json.loads(data)
  File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Error: Process completed with exit code 1.

@estesp
Copy link
Copy Markdown
Member

estesp commented Oct 12, 2023

I noticed that too and assumed it was simply in different states of "broken" at those moments; the python error seems to just be an unhandled/unexpected result of not getting valid data in the HTTPS response from the apt repo: e.g. "in get_info_from_https"

@estesp
Copy link
Copy Markdown
Member

estesp commented Oct 12, 2023

Maybe they are updating their HTTP/2 servers for you-know-what CVE 😅

@thaJeztah
Copy link
Copy Markdown
Member Author

Ah, thanks; guess it's just a glitch then (hopefully); just failing in a slightly different stage of things.

I was also mistaken; apt-key is the one that was deprecated, not add-apt-repository

@thaJeztah
Copy link
Copy Markdown
Member Author

Hm.. so, I guess I completely interpret the original intent in reverse; and "build binaries with go1.21" had the intent to "verify binaries also work with go1.21", but not to build (and release) binaries with go1.21.

Let me move this to draft to prevent trigger-happy merges, pending further discussion

@thaJeztah thaJeztah marked this pull request as draft October 13, 2023 15:19
@dmcgowan
Copy link
Copy Markdown
Member

Hm.. so, I guess I completely interpret the original intent in reverse; and "build binaries with go1.21" had the intent to "verify binaries also work with go1.21", but not to build (and release) binaries with go1.21.

We should probably define the go version more clearly for our branches. I think it is fair to say "on main, we always update to the latest after .1 release" or something, then on release branches we only update the go version when it is about to go end of life (when n+2 is released).

@thaJeztah
Copy link
Copy Markdown
Member Author

For main branch, I think (unless we're very close to a new release / release branch), it would be fine to update immediately; that way main can have some burn-in time, and we can verify there's no oddities. Of course that's assuming a Go .1 release will be done before we cut the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants