Merge latest main branch into feature/openvmm - #42020

Co-authored-by: WSL localization <[email protected]>

…port) (#41546)

The current logic uses a fixed 4KiB buffer for stdio relay on the Windows side. And uses an initial 4KiB buffer for stdio relay on the Linux side, which grows only if the message won't fit. This can severely limit the relay performance in some situations. For example, in #41572, when redirecting stdout to a SMB share.

This PR increases thedefault relay buffer size to 64KiB. Which shows significant performance improvements according to buffer size tests.

Currently the stdout and stderr relay tracks the output offset separately. And when redirected to the same file, the writes could overlap.

This PR uses the append mode for overlap io writes instead of the separate offsets in the relay. So, the file write offset is correctly tracked by the system.

The installer log files were collected before the log collection starts. Which will miss the user repro.

This PR moves it after the user repro.

Co-authored-by: Copilot <[email protected]>

Co-authored-by: WSL localization <[email protected]>

In mirrored mode, handle route update as add instead of replace
Remove usage of route update as it can overwrite routes on other interfaces. add tests that verify host routing changes are correctly reflected in Linux

---------

Co-authored-by: Catalin-Emil Fetoiu <[email protected]>

…PN is running (#41638)

* Don't fail the UnitTests::Warnings test case if GlobalSecureAccess VPN is running

* Cleanup diff

* Fix test

* Wait for wslservice to be started when running tests

* Fail on timeout

* Improve check

* Format

In 2.9.8, the distro processes are separated into their own cgroups. But they remain in the same cgroup namespace. That caused compatibility issues with softwares that assume a fixed systemd cgroup layout. For example, rootless docker and nerdctl.
Fixes on Moby and nertctl are being worked on. However, to avoid issues with other software, WSL's cgroup handling should also be improved.

This PR creates new cgroup namespaces for the distros. So, to the non-critical distro processes, the systemd cgroup layout stays the same as before.
This PR also introduces a cgroup structure change to accomplish this. The systemd init is moved from wsl-user/distro-N/systemd to wsl-user/distro-N. And the initialization of the distro-N controllers is handled by systemd instead.
The processes are also moved into the non-systemd cgroup in systemdless distros. This makes sure that sub-group controllers can be enabled in the distro root.
Cgroup v2 is now enforced when distro isolation is enabled. Instead of constructing an unusable cgroup v1 layout when cgroup v1 and distro isolation are both enabled.

Before this change, an additional 10ms delay was added in the port tracker loop presumably to make the main loop slower than the worker loop. So, the worker result does not get super dated. This is not reliable. And the 10ms delay also slows down every consecutive bind call.

This PR refactors the thread synchronization method. So, the timing between those two loops is more deterministic. And removes the performance penalty for all bind calls.

Co-authored-by: WSL localization <[email protected]>

Co-authored-by: Copilot <[email protected]>

…ncorrectly disabled- #41648 (#41648)

Co-authored-by: Catalin-Emil Fetoiu <[email protected]>

…41619)

* Move objects shared by wslc.exe and the tests to src/windows/common

Co-authored-by: Copilot <[email protected]>

* Link yaml-cpp, advapi32 and Crypt32 into common

Co-authored-by: Copilot <[email protected]>

* Drop yaml-cpp, advapi32 and crypt32 from wslclib now that common provides them

Co-authored-by: Copilot <[email protected]>

* Move wslc code in common under common/wslc and namespace CLI types as wslc::cli

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>

* Save state

* Use proper arrays

* Format

* Add new tests

* Cleanup tests

* Cleanup diff

* Format

* Apply PR feedback

* Apply PR feedback

* Format

* Apply PR feedback

* Move logging call

* Apply PR feedback

* Apply PR feedback

Co-authored-by: WSL localization <[email protected]>

* Add WSLC network lifecycle events

* Localize pending network prune errors

* Fix network operation wait ownership and cleanup ordering

* Fix prune event correlation after timed-out

* Fix network event rollback and timeout recovery

* Format

* Forward Docker network events directly

* Restore lock_guard after removing event waits

* fix test

* align event stream test helpers after merge

* feedback

…n init messages (#41664)

Co-authored-by: WSL localization <[email protected]>

When parsing the config file, non-ascii utf-16 values are casted into char. This will mangle non-ascii values in the config file. For example, a custom kernel path containing Chinese characters will cause WSL_E_CUSTOM_KERNEL_NOT_FOUND.

This PR removes the faulty static cast and stores the config value in wstring instead of string. So non-ascii values are preserved.

This is a follow up to PR #41512

Remove the redundant restore cgroup namespace code.
Simplify the cgroup check logic by moving the retry inside the debug shell.
Add a new systemd boot timeout test.

* Match console detach sequence on full UTF-16 chars

Co-authored-by: Copilot <[email protected]>

* Add console detach unicode low-byte test

Co-authored-by: Copilot <[email protected]>

* Move detach unicode test to wslc pseudoconsole E2E

Co-authored-by: Copilot <[email protected]>

* Write detach test unicode pair separately from marker

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>

Co-authored-by: WSL notice <[email protected]>

* fix start up parsing regression

* Fix distro config parsing logic

* Add unit test

Co-authored-by: WSL notice <[email protected]>

* Fix concurrent resource access in WSL1 instance during OOBE

* Don't get stuck if the distro gets stuck

…gs when stuck (#42001)

…nationBehavior (#42004)

…ially exit while references are still held by clients (#42000)

* Save state

* Mark pruned containers as exited

* Revert "Mark pruned containers as exited"

This reverts commit 91af5d2, including its test changes.

* Load container images

* Use a GIT reference

* Fix potential use free in WSLCContainer

Co-authored-by: Copilot <[email protected]>

…sdk (#41681)

* add --cap-add and --cap-drop options

* avoid repeating the warning on retry

* update docs

* fix localization

* update component check logic to avoid mismatch

* block container run and create when cap add is disabled

* Use version check instead of interface check

* allow current version in runtime version check

* resolve compatibility unrelated comments

* change compatibility design

* resolve comment

---------

Co-authored-by: Feng Wang (from Dev Box) <[email protected]>