Auto merge of #113411 - unikraft:unikraft, r=wesleywiser

bors · bors · commit d24c4da1d604 · 2023-07-25T03:41:56.000Z
Add `x86_64-unikraft-linux-musl` target This introduces `x86_64-unikraft-linux-musl` as the first Rust target for the [Unikraft] Unikernel Development Kit. [Unikraft]: https://unikraft.org/ Unikraft imitates Linux and uses musl as libc. It is extremely configurable, and does not even provide a `poll` implementation or a network stack, unless enabled by the end user who compiles the application. Our approach for integrating the build process with `rustc` is to hide the build process as well as the actual final linking step behind a linker-shim (`kraftld`, see unikraft/kraftkit#612). ## Tier 3 target policy > - A tier 3 target must have a designated developer or developers (the "target > maintainers") on record to be CCed when issues arise regarding the target. > (The mechanism to track and CC such developers may evolve over time.) I will be the target maintainer. > - Targets must use naming consistent with any existing targets; for instance, a > target for the same CPU or OS as an existing Rust target should use the same > name for that CPU or OS. Targets should normally use the same names and > naming conventions as used elsewhere in the broader ecosystem beyond Rust > (such as in other toolchains), unless they have a very good reason to > diverge. Changing the name of a target can be highly disruptive, especially > once the target reaches a higher tier, so getting the name right is important > even for a tier 3 target. > - Target names should not introduce undue confusion or ambiguity unless > absolutely necessary to maintain ecosystem compatibility. For example, if > the name of the target makes people extremely likely to form incorrect > beliefs about what it targets, the name should be changed or augmented to > disambiguate it. > - If possible, use only letters, numbers, dashes and underscores for the name. > Periods (`.`) are known to cause issues in Cargo. The target name `x86_64-unikraft-linux-musl` was derived from `x86_64-unknown-linux-musl`, setting Unikraft as vendor. Unikraft exactly imitates Linux + musl. > - Tier 3 targets may have unusual requirements to build or use, but must not > create legal issues or impose onerous legal terms for the Rust project or for > Rust developers or users. > - The target must not introduce license incompatibilities. > - Anything added to the Rust repository must be under the standard Rust > license (`MIT OR Apache-2.0`). > - The target must not cause the Rust tools or libraries built for any other > host (even when supporting cross-compilation to the target) to depend > on any new dependency less permissive than the Rust licensing policy. This > applies whether the dependency is a Rust crate that would require adding > new license exceptions (as specified by the `tidy` tool in the > rust-lang/rust repository), or whether the dependency is a native library > or binary. In other words, the introduction of the target must not cause a > user installing or running a version of Rust or the Rust tools to be > subject to any new license requirements. > - Compiling, linking, and emitting functional binaries, libraries, or other > code for the target (whether hosted on the target itself or cross-compiling > from another target) must not depend on proprietary (non-FOSS) libraries. > Host tools built for the target itself may depend on the ordinary runtime > libraries supplied by the platform and commonly used by other applications > built for the target, but those libraries must not be required for code > generation for the target; cross-compilation to the target must not require > such libraries at all. For instance, `rustc` built for the target may > depend on a common proprietary C runtime library or console output library, > but must not depend on a proprietary code generation library or code > optimization library. Rust's license permits such combinations, but the > Rust project has no interest in maintaining such combinations within the > scope of Rust itself, even at tier 3. > - "onerous" here is an intentionally subjective term. At a minimum, "onerous" > legal/licensing terms include but are *not* limited to: non-disclosure > requirements, non-compete requirements, contributor license agreements > (CLAs) or equivalent, "non-commercial"/"research-only"/etc terms, > requirements conditional on the employer or employment of any particular > Rust developers, revocable terms, any requirements that create liability > for the Rust project or its developers or users, or any requirements that > adversely affect the livelihood or prospects of the Rust project or its > developers or users. No dependencies were added to Rust. Requirements for linking are [Unikraft] and [KraftKit] (both BSD-3-Clause), but none of these are added to Rust. [KraftKit]: https://github.com/unikraft/kraftkit > - Neither this policy nor any decisions made regarding targets shall create any > binding agreement or estoppel by any party. If any member of an approving > Rust team serves as one of the maintainers of a target, or has any legal or > employment requirement (explicit or implicit) that might affect their > decisions regarding a target, they must recuse themselves from any approval > decisions regarding the target's tier status, though they may otherwise > participate in discussions. > - This requirement does not prevent part or all of this policy from being > cited in an explicit contract or work agreement (e.g. to implement or > maintain support for a target). This requirement exists to ensure that a > developer or team responsible for reviewing and approving a target does not > face any legal threats or obligations that would prevent them from freely > exercising their judgment in such approval, even if such judgment involves > subjective matters or goes beyond the letter of these requirements. Understood. I am not a member of a Rust team. > - Tier 3 targets should attempt to implement as much of the standard libraries > as possible and appropriate (`core` for most targets, `alloc` for targets > that can support dynamic memory allocation, `std` for targets with an > operating system or equivalent layer of system-provided functionality), but > may leave some code unimplemented (either unavailable or stubbed out as > appropriate), whether because the target makes it impossible to implement or > challenging to implement. The authors of pull requests are not obligated to > avoid calling any portions of the standard library on the basis of a tier 3 > target not implementing those portions. Understood. `std` is supported. > - The target must provide documentation for the Rust community explaining how > to build for the target, using cross-compilation if possible. If the target > supports running binaries, or running tests (even if they do not pass), the > documentation must explain how to run such binaries or tests for the target, > using emulation if possible or dedicated hardware if necessary. Building is described in the platform support doc. It will be updated once proper `kraftld` support has landed. > - Tier 3 targets must not impose burden on the authors of pull requests, or > other developers in the community, to maintain the target. In particular, > do not post comments (automated or manual) on a PR that derail or suggest a > block on the PR based on a tier 3 target. Do not send automated messages or > notifications (via any medium, including via ``@`)` to a PR author or others > involved with a PR regarding a tier 3 target, unless they have opted into > such messages. > - Backlinks such as those generated by the issue/PR tracker when linking to > an issue or PR are not considered a violation of this policy, within > reason. However, such messages (even on a separate repository) must not > generate notifications to anyone involved with a PR who has not requested > such notifications. Understood. > - Patches adding or updating tier 3 targets must not break any existing tier 2 > or tier 1 target, and must not knowingly break another tier 3 target without > approval of either the compiler team or the maintainers of the other tier 3 > target. > - In particular, this may come up when working on closely related targets, > such as variations of the same architecture with different features. Avoid > introducing unconditional uses of features that another variation of the > target may not have; use conditional compilation or runtime detection, as > appropriate, to let each target run code supported by that target. I don't think this PR breaks anything. r? compiler-team
diff --git a/compiler/rustc_target/src/spec/mod.rs b/compiler/rustc_target/src/spec/mod.rs
@@ -85,6 +85,7 @@ mod solaris_base;
 mod solid_base;
 mod thumb_base;
 mod uefi_msvc_base;
+mod unikraft_linux_musl_base;
 mod vxworks_base;
 mod wasm_base;
 mod windows_gnu_base;
@@ -1421,6 +1422,8 @@ supported_targets! {
     ("riscv64gc-unknown-hermit", riscv64gc_unknown_hermit),
     ("x86_64-unknown-hermit", x86_64_unknown_hermit),
 
+    ("x86_64-unikraft-linux-musl", x86_64_unikraft_linux_musl),
+
     ("riscv32i-unknown-none-elf", riscv32i_unknown_none_elf),
     ("riscv32im-unknown-none-elf", riscv32im_unknown_none_elf),
     ("riscv32imc-unknown-none-elf", riscv32imc_unknown_none_elf),
diff --git a/compiler/rustc_target/src/spec/unikraft_linux_musl_base.rs b/compiler/rustc_target/src/spec/unikraft_linux_musl_base.rs
@@ -0,0 +1,15 @@
+use crate::spec::{cvs, PanicStrategy, RelocModel, TargetOptions};
+
+pub fn opts() -> TargetOptions {
+    TargetOptions {
+        os: "linux".into(),
+        env: "musl".into(),
+        vendor: "unikraft".into(),
+        linker: Some("kraftld".into()),
+        relocation_model: RelocModel::Static,
+        families: cvs!["unix"],
+        has_thread_local: true,
+        panic_strategy: PanicStrategy::Abort,
+        ..Default::default()
+    }
+}
diff --git a/compiler/rustc_target/src/spec/x86_64_unikraft_linux_musl.rs b/compiler/rustc_target/src/spec/x86_64_unikraft_linux_musl.rs
@@ -0,0 +1,19 @@
+use crate::spec::{Cc, LinkerFlavor, Lld, StackProbeType, Target, TargetOptions};
+
+pub fn target() -> Target {
+    Target {
+        llvm_target: "x86_64-unknown-linux-musl".into(),
+        pointer_width: 64,
+        arch: "x86_64".into(),
+        data_layout: "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+            .into(),
+        options: TargetOptions {
+            cpu: "x86-64".into(),
+            plt_by_default: false,
+            pre_link_args: TargetOptions::link_args(LinkerFlavor::Gnu(Cc::Yes, Lld::No), &["-m64"]),
+            max_atomic_width: Some(64),
+            stack_probes: StackProbeType::X86,
+            ..super::unikraft_linux_musl_base::opts()
+        },
+    }
+}
diff --git a/library/std/src/sys/unix/mod.rs b/library/std/src/sys/unix/mod.rs
@@ -110,6 +110,11 @@ pub unsafe fn init(argc: isize, argv: *const *const u8, sigpipe: u8) {
             while libc::poll(pfds.as_mut_ptr(), 3, 0) == -1 {
                 match errno() {
                     libc::EINTR => continue,
+                    #[cfg(target_vendor = "unikraft")]
+                    libc::ENOSYS => {
+                        // Not all configurations of Unikraft enable `LIBPOSIX_EVENT`.
+                        break 'poll;
+                    }
                     libc::EINVAL | libc::EAGAIN | libc::ENOMEM => {
                         // RLIMIT_NOFILE or temporary allocation failures
                         // may be preventing use of poll(), fall back to fcntl
@@ -165,7 +170,14 @@ pub unsafe fn init(argc: isize, argv: *const *const u8, sigpipe: u8) {
     }
 
     unsafe fn reset_sigpipe(#[allow(unused_variables)] sigpipe: u8) {
-        #[cfg(not(any(target_os = "emscripten", target_os = "fuchsia", target_os = "horizon")))]
+        #[cfg(not(any(
+            target_os = "emscripten",
+            target_os = "fuchsia",
+            target_os = "horizon",
+            // Unikraft's `signal` implementation is currently broken:
+            // https://github.com/unikraft/lib-musl/issues/57
+            target_vendor = "unikraft",
+        )))]
         {
             // We don't want to add this as a public type to std, nor do we
             // want to `include!` a file from the compiler (which would break
diff --git a/src/bootstrap/compile.rs b/src/bootstrap/compile.rs
@@ -261,7 +261,7 @@ fn copy_self_contained_objects(
     // to using gcc from a glibc-targeting toolchain for linking.
     // To do that we have to distribute musl startup objects as a part of Rust toolchain
     // and link with them manually in the self-contained mode.
-    if target.contains("musl") {
+    if target.contains("musl") && !target.contains("unikraft") {
         let srcdir = builder.musl_libdir(target).unwrap_or_else(|| {
             panic!("Target {:?} does not have a \"musl-libdir\" key", target.triple)
         });
diff --git a/src/bootstrap/lib.rs b/src/bootstrap/lib.rs
@@ -131,6 +131,8 @@ const EXTRA_CHECK_CFGS: &[(Option<Mode>, &'static str, Option<&[&'static str]>)]
     (Some(Mode::Std), "freebsd13", None),
     (Some(Mode::Std), "backtrace_in_libstd", None),
     /* Extra values not defined in the built-in targets yet, but used in std */
+    // #[cfg(bootstrap)]
+    (Some(Mode::Std), "target_vendor", Some(&["unikraft"])),
     (Some(Mode::Std), "target_env", Some(&["libnx"])),
     // (Some(Mode::Std), "target_os", Some(&[])),
     // #[cfg(bootstrap)] mips32r6, mips64r6
diff --git a/src/bootstrap/sanity.rs b/src/bootstrap/sanity.rs
@@ -206,7 +206,7 @@ than building it.
         }
 
         // Make sure musl-root is valid
-        if target.contains("musl") {
+        if target.contains("musl") && !target.contains("unikraft") {
             // If this is a native target (host is also musl) and no musl-root is given,
             // fall back to the system toolchain in /usr before giving up
             if build.musl_root(*target).is_none() && build.config.build == *target {
diff --git a/src/doc/rustc/src/SUMMARY.md b/src/doc/rustc/src/SUMMARY.md
@@ -42,6 +42,7 @@
     - [sparc-unknown-none-elf](./platform-support/sparc-unknown-none-elf.md)
     - [*-pc-windows-gnullvm](platform-support/pc-windows-gnullvm.md)
     - [\*-nto-qnx-\*](platform-support/nto-qnx.md)
+    - [*-unikraft-linux-musl](platform-support/unikraft-linux-musl.md)
     - [*-unknown-hermit](platform-support/hermit.md)
     - [\*-unknown-netbsd\*](platform-support/netbsd.md)
     - [*-unknown-openbsd](platform-support/openbsd.md)
diff --git a/src/doc/rustc/src/platform-support.md b/src/doc/rustc/src/platform-support.md
@@ -326,6 +326,7 @@ target | std | host | notes
 [`x86_64-pc-windows-gnullvm`](platform-support/pc-windows-gnullvm.md) | ✓ | ✓ |
 `x86_64-pc-windows-msvc` | * |  | 64-bit Windows XP support
 `x86_64-sun-solaris` | ? |  | Deprecated target for 64-bit Solaris 10/11, illumos
+[`x86_64-unikraft-linux-musl`](platform-support/unikraft-linux-musl.md) | ✓ |   | 64-bit Unikraft with musl
 `x86_64-unknown-dragonfly` | ✓ | ✓ | 64-bit DragonFlyBSD
 `x86_64-unknown-haiku` | ✓ | ✓ | 64-bit Haiku
 [`x86_64-unknown-hermit`](platform-support/hermit.md) | ✓ |  | x86_64 Hermit
diff --git a/src/doc/rustc/src/platform-support/unikraft-linux-musl.md b/src/doc/rustc/src/platform-support/unikraft-linux-musl.md
@@ -0,0 +1,67 @@
+# `*-unikraft-linux-musl`
+
+**Tier: 3**
+
+Targets for the [Unikraft] Unikernel Development Kit (with musl).
+
+[Unikraft]: https://unikraft.org/
+
+Target triplets available so far:
+
+- `x86_64-unikraft-linux-musl`
+
+## Target maintainers
+
+- Martin Kröning ([@mkroening](https://github.com/mkroening))
+
+## Requirements
+
+These targets only support cross-compilation.
+The targets do support std.
+
+Unikraft pretends to behave exactly like Linux.
+How much of that functionality is available depends on the individual unikernel configuration.
+For example, the basic Unikraft + musl config does not support `poll` or networking out of the box.
+That functionality requires enabling [`LIBPOSIX_EVENT`] or [lwIP] respectively.
+
+[`LIBPOSIX_EVENT`]: https://github.com/unikraft/unikraft/blob/RELEASE-0.13.1/lib/posix-event/Config.uk
+[lwIP]: https://github.com/unikraft/lib-lwip
+
+The Unikraft targets follow Linux's `extern "C"` calling convention.
+
+For these targets, `rustc` does not perform the final linking step.
+Instead, the Unikraft build system will produce the final Unikernel image for the selected platform (e.g., KVM, Linux user space, and Xen).
+
+## Building the targets
+
+You can build Rust with support for the targets by adding it to the `target` list in `config.toml`:
+
+```toml
+[build]
+build-stage = 1
+target = [ "x86_64-unikraft-linux-musl" ]
+```
+
+## Building Rust programs
+
+Rust does not yet ship pre-compiled artifacts for these targets.
+To compile for these targets, you will either need to build Rust with the targets enabled
+(see “Building the targets” above), or build your own copy of `core` by using `build-std` or similar.
+
+Linking requires a [KraftKit] shim.
+See [unikraft/kraftkit#612] for more information.
+
+[KraftKit]: https://github.com/unikraft/kraftkit
+[unikraft/kraftkit#612]: https://github.com/unikraft/kraftkit/issues/612
+
+## Testing
+
+The targets do support running binaries in the form of unikernel images.
+How the unikernel image is run depends on the specific platform (e.g., KVM, Linux user space, and Xen).
+The targets do not support running the Rust test suite.
+
+## Cross-compilation toolchains and C code
+
+The targets do support C code.
+To build compatible C code, you have to use the same compiler and flags as does the Unikraft build system for your specific configuration.
+The easiest way to achieve that, is to build the C code with the Unikraft build system when building your unikernel image.
diff --git a/src/tools/build-manifest/src/main.rs b/src/tools/build-manifest/src/main.rs
@@ -148,6 +148,7 @@ static TARGETS: &[&str] = &[
     "x86_64-pc-windows-msvc",
     "x86_64-sun-solaris",
     "x86_64-pc-solaris",
+    "x86_64-unikraft-linux-musl",
     "x86_64-unknown-freebsd",
     "x86_64-unknown-illumos",
     "x86_64-unknown-linux-gnu",

Original file line number	Diff line number	Diff line change
`@@ -206,7 +206,7 @@ than building it.`
`206`	`206`	`}`
`207`	`207`
`208`	`208`	`// Make sure musl-root is valid`
`209`		`- if target.contains("musl") {`
	`209`	`+ if target.contains("musl") && !target.contains("unikraft") {`
`210`	`210`	`// If this is a native target (host is also musl) and no musl-root is given,`
`211`	`211`	`// fall back to the system toolchain in /usr before giving up`
`212`	`212`	`if build.musl_root(target).is_none() && build.config.build == target {`