… rlimit_test.go

Tests in rlimit_test.go exist to test the behavior of automatically
bumping RLIMIT_NOFILE on Unix implemented in rlimit.go (issue #46279),
with darwin-specific behavior split out into rlimit_darwin.go and
the rest left empty in rlimit_stub.go.

Since the behavior happens only on Unix, it doesn't make sense to test
it on other platforms. Copy rlimit.go's 'unix' build constraint to
rlimit_test.go to accomplish that.

Leave out the simplification of the build constraint in rlimit_stub.go
so that this CL remains a test-only fix.

In particular, this fixes a problem where TestOpenFileLimit was
failing in some environments when testing the wasip1/wasm port.
The RLIMIT_NOFILE bumping behavior isn't implemented there, so
the test was testing the environment and not the Go project.

Updates #46279.
For #61116.
Fixes #63994.

Change-Id: Ic993f9cfc021d4cda4fe3d7fed8e2e180f78a2ca
Cq-Include-Trybots: luci.golang.try:go1.21-wasip1-wasm_wasmtime
Reviewed-on: https://go-review.googlesource.com/c/go/+/539435
Reviewed-by: Johan Brandhorst-Satzkorn <[email protected]>
Reviewed-by: Bryan Mills <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Dmitri Shuralyov <[email protected]>
Auto-Submit: Dmitri Shuralyov <[email protected]>
(cherry picked from commit b7cbcf0)
Reviewed-on: https://go-review.googlesource.com/c/go/+/540615
Reviewed-by: Heschi Kreinick <[email protected]>
Auto-Submit: Heschi Kreinick <[email protected]>

…in dead store elimination

Update #63743

Change-Id: I163c6038c13d974dc0ca9f02144472bc05331826
Reviewed-on: https://go-review.googlesource.com/c/go/+/538595
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: David Chase <[email protected]>
Reviewed-by: Keith Randall <[email protected]>
(cherry picked from commit 43b57b8)
Reviewed-on: https://go-review.googlesource.com/c/go/+/538857
Auto-Submit: Heschi Kreinick <[email protected]>
Reviewed-by: Heschi Kreinick <[email protected]>

…s after the nil check

Have nil checks return a pointer that is known non-nil. Users of
that pointer can use the result, ensuring that they are ordered
after the nil check itself.

The order dependence goes away after scheduling, when we've fixed
an order. At that point we move uses back to the original pointer
so it doesn't change regalloc any.

This prevents pointer arithmetic on nil from being spilled to the
stack and then observed by a stack scan.

Fixes #63743

Change-Id: I1a5fa4f2e6d9000d672792b4f90dfc1b7b67f6ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/537775
Reviewed-by: David Chase <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Keith Randall <[email protected]>
(cherry picked from commit 962ccbe)
Reviewed-on: https://go-review.googlesource.com/c/go/+/538717
Auto-Submit: Heschi Kreinick <[email protected]>
Reviewed-by: Heschi Kreinick <[email protected]>

… splice to avoid inconsistency with O_NONBLOCK

Fixes #63801
Updates #59041
Updates #63795

Details: #59041 (comment)

Change-Id: Id3fc1df6d86b7c4cc383d09f9465fa8f4cc2a559
Reviewed-on: https://go-review.googlesource.com/c/go/+/536015
Reviewed-by: Bryan Mills <[email protected]>
Reviewed-by: Ian Lance Taylor <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Auto-Submit: Ian Lance Taylor <[email protected]>
(cherry picked from commit 40cdf69)
Reviewed-on: https://go-review.googlesource.com/c/go/+/538117
Auto-Submit: Heschi Kreinick <[email protected]>
Reviewed-by: Mauri de Souza Meneguzzo <[email protected]>
Reviewed-by: Heschi Kreinick <[email protected]>

…lar in Stat and Lstat

Prior to CL 460595, Lstat reported most reparse points as regular
files. However, reparse points can in general implement unusual
behaviors (consider IO_REPARSE_TAG_AF_UNIX or IO_REPARSE_TAG_LX_CHR),
and Windows allows arbitrary user-defined reparse points, so in
general we must not assume that an unrecognized reparse tag represents
a regular file; in CL 460595, we began marking them as irregular.

As it turns out, the Data Deduplication service on Windows Server runs
an Optimization job that turns regular files into reparse files with
the tag IO_REPARSE_TAG_DEDUP. Those files still behave more-or-less
like regular files, in that they have well-defined sizes and support
random-access reads and writes, so most programs can treat them as
regular files without difficulty. However, they are still reparse
files: as a result, on servers with the Data Deduplication service
enabled, files could arbitrarily change from “regular” to “irregular”
without explicit user intervention.

Since dedup files are converted in the background and otherwise behave
like regular files, this change adds a special case to report DEDUP
reparse points as regular.

Fixes #63764.
Updates #63429.

No test because to my knowledge we don't have any Windows builders
that have the deduplication service enabled, nor do we have a way to
reliably guarantee the existence of an IO_REPARSE_TAG_DEDUP file.

(In theory we could add a builder with the service enabled on a
specific volume, write a test that encodes knowledge of that volume,
and use the GO_BUILDER_NAME environment variable to run that test only
on the specially-configured builders. However, I don't currently have
the bandwidth to reconfigure the builders in this way, and given the
simplicity of the change I think it is unlikely to regress
accidentally.)

Change-Id: I649e7ef0b67e3939a980339ce7ec6a20b31b23a1
Cq-Include-Trybots: luci.golang.try:go1.21-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/538218
Reviewed-by: Alex Brainman <[email protected]>
Reviewed-by: David Chase <[email protected]>
Auto-Submit: Heschi Kreinick <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>

…lchains if called with explicit arguments

Fixes #62055.
Updates #62054.

Change-Id: I4ea24070f7d9aa4964c2f215836602068058f718
(cherry picked from CL 540779 and CL 537480)
Reviewed-on: https://go-review.googlesource.com/c/go/+/539697
Auto-Submit: Heschi Kreinick <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Michael Matloob <[email protected]>

…ed repo does not support a secure protocol

Updates #63845.
Fixes #63973.

Change-Id: If86d6b13d3b55877b35c087112bd76388c9404b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/539321
Reviewed-by: Michael Matloob <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Roland Shoemaker <[email protected]>
Auto-Submit: Bryan Mills <[email protected]>
(cherry picked from commit be26ae1)
Reviewed-on: https://go-review.googlesource.com/c/go/+/540257
Auto-Submit: Dmitri Shuralyov <[email protected]>
Reviewed-by: Dmitri Shuralyov <[email protected]>

… Windows

While fixing several bugs in path handling on Windows,
beginning with \\?\.

Prior to #540277, VolumeName considered the first path component
after the \\?\ prefix to be part of the volume name.
After, it considered only the \\? prefix to be the volume name.

Restore the previous behavior.

For #64028.
Fixes #64041.

Change-Id: I6523789e61776342800bd607fb3f29d496257e68
Reviewed-on: https://go-review.googlesource.com/c/go/+/541175
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Roland Shoemaker <[email protected]>
(cherry picked from commit eda42f7)
Reviewed-on: https://go-review.googlesource.com/c/go/+/541521
Reviewed-by: Damien Neil <[email protected]>
Auto-Submit: Dmitri Shuralyov <[email protected]>
Reviewed-by: Dmitri Shuralyov <[email protected]>

…ProcessPrng

RtlGenRandom is a semi-undocumented API, also known as
SystemFunction036, which we use to generate random data on Windows.
It's definition, in cryptbase.dll, is an opaque wrapper for the
documented API ProcessPrng. Instead of using RtlGenRandom, switch to
using ProcessPrng, since the former is simply a wrapper for the latter,
there should be no practical change on the user side, other than a minor
change in the DLLs we load.

Updates #53192
Fixes #64413

Change-Id: Ie6891bf97b1d47f5368cccbe92f374dba2c2672a
Reviewed-on: https://go-review.googlesource.com/c/go/+/536235
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Quim Muntal <[email protected]>
Auto-Submit: Roland Shoemaker <[email protected]>
Reviewed-by: Dmitri Shuralyov <[email protected]>
(cherry picked from commit 693def1)
Reviewed-on: https://go-review.googlesource.com/c/go/+/545355
Auto-Submit: Dmitri Shuralyov <[email protected]>

… callees on the systemstack

These functions acquire the heap lock. If they're not called on the
systemstack, a stack growth could cause a self-deadlock since stack
growth may allocate memory from the page heap.

This has been a problem for a while. If this is what's plaguing the
ppc64 port right now, it's very surprising (and probably just
coincidental) that it's showing up now.

For #64050.
For #64062.
For #64067.
Fixes #64073.

Change-Id: I2b95dc134d17be63b9fe8f7a3370fe5b5438682f
Reviewed-on: https://go-review.googlesource.com/c/go/+/541635
LUCI-TryBot-Result: Go LUCI <[email protected]>
Run-TryBot: Michael Knyszek <[email protected]>
Auto-Submit: Michael Knyszek <[email protected]>
TryBot-Result: Gopher Robot <[email protected]>
Reviewed-by: Michael Pratt <[email protected]>
Reviewed-by: Paul Murphy <[email protected]>
(cherry picked from commit 5f08b44)
Reviewed-on: https://go-review.googlesource.com/c/go/+/541955
Reviewed-by: Dmitri Shuralyov <[email protected]>
Reviewed-by: Dmitri Shuralyov <[email protected]>
Auto-Submit: Dmitri Shuralyov <[email protected]>

…tch disjointed loop headers

Fix #63984

parseIndVar, prove and maybe more are on the assumption that the loop header
is a single block. This can be wrong, ensure we don't match theses cases we
don't know how to handle.

In the future we could update them so that they know how to handle such cases
but theses cases seems rare so I don't think the value would be really high.
We could also run a loop canonicalization pass first which could handle this.

The repro case looks weird because I massaged it so it would crash with the
previous compiler.

Change-Id: I4aa8afae9e90a17fa1085832250fc1139c97faa6
Reviewed-on: https://go-review.googlesource.com/c/go/+/539977
Reviewed-by: Heschi Kreinick <[email protected]>
Reviewed-by: Keith Randall <[email protected]>
Reviewed-by: Keith Randall <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
(cherry picked from commit 8b4e125)
Reviewed-on: https://go-review.googlesource.com/c/go/+/540535
Reviewed-by: Jorropo <[email protected]>
Reviewed-by: Mauri de Souza Meneguzzo <[email protected]>
Reviewed-by: Dmitri Shuralyov <[email protected]>
Reviewed-by: Dmitri Shuralyov <[email protected]>
Auto-Submit: Dmitri Shuralyov <[email protected]>
Reviewed-by: Michael Knyszek <[email protected]>

The chunked transfer encoding adds some overhead to
the content transferred. When writing one byte per
chunk, for example, there are five bytes of overhead
per byte of data transferred: "1\r\nX\r\n" to send "X".

Chunks may include "chunk extensions",
which we skip over and do not use.
For example: "1;chunk extension here\r\nX\r\n".

A malicious sender can use chunk extensions to add
about 4k of overhead per byte of data.
(The maximum chunk header line size we will accept.)

Track the amount of overhead read in chunked data,
and produce an error if it seems excessive.

Updates #64433
Fixes #64435
Fixes CVE-2023-39326

Change-Id: I40f8d70eb6f9575fb43f506eb19132ccedafcf39
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2076135
Reviewed-by: Tatiana Bradley <[email protected]>
Reviewed-by: Roland Shoemaker <[email protected]>
(cherry picked from commit 3473ae72ee66c60744665a24b2fde143e8964d4f)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2095408
Run-TryBot: Roland Shoemaker <[email protected]>
Reviewed-by: Damien Neil <[email protected]>
Reviewed-on: https://go-review.googlesource.com/c/go/+/547356
Reviewed-by: Dmitri Shuralyov <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>

Change-Id: I203463900d2c63bc398c22593d0908c4f33a6990
Reviewed-on: https://go-review.googlesource.com/c/go/+/547416
Reviewed-by: Carlos Amedee <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Auto-Submit: Gopher Robot <[email protected]>
Reviewed-by: Dmitri Shuralyov <[email protected]>