Skip to content

Commit 1cb8a18

Browse files
rust-timerrust-timer
authored
Unrolled build for rust-lang#125005
Rollup merge of rust-lang#125005 - RalfJung:miri-sync, r=RalfJung Miri subtree update r? `@ghost`
2 parents 35c5e67 + 79a85d4 commit 1cb8a18

File tree

156 files changed

+1182
-1166
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

156 files changed

+1182
-1166
lines changed

‎src/tools/miri/.gitignore

+1-1
Original file line numberDiff line numberDiff line change
@@ -9,5 +9,5 @@ tex/*/out
99
perf.data
1010
perf.data.old
1111
flamegraph.svg
12-
tests/extern-so/libtestlib.so
12+
tests/native-lib/libtestlib.so
1313
.auto-*

‎src/tools/miri/CONTRIBUTING.md

+44-7
Original file line numberDiff line numberDiff line change
@@ -72,14 +72,14 @@ For example:
7272

7373
You can (cross-)run the entire test suite using:
7474

75-
```
75+
```sh
7676
./miri test
77-
MIRI_TEST_TARGET=i686-unknown-linux-gnu ./miri test
77+
./miri test --target i686-unknown-linux-gnu
7878
```
7979

8080
`./miri test FILTER` only runs those tests that contain `FILTER` in their filename (including the
81-
base directory, e.g. `./miri test fail` will run all compile-fail tests). These filters are passed
82-
to `cargo test`, so for multiple filters you need to use `./miri test -- FILTER1 FILTER2`.
81+
base directory, e.g. `./miri test fail` will run all compile-fail tests). Multiple filters are
82+
supported: `./miri test FILTER1 FILTER2` runs all tests that contain either string.
8383

8484
#### Fine grained logging
8585

@@ -139,9 +139,8 @@ and then you can use it as if it was installed by `rustup` as a component of the
139139
in the `miri` toolchain's sysroot to prevent conflicts with other toolchains.
140140
The Miri binaries in the `cargo` bin directory (usually `~/.cargo/bin`) are managed by rustup.
141141

142-
There's a test for the cargo wrapper in the `test-cargo-miri` directory; run
143-
`./run-test.py` in there to execute it. Like `./miri test`, this respects the
144-
`MIRI_TEST_TARGET` environment variable to execute the test for another target.
142+
There's a test for the cargo wrapper in the `test-cargo-miri` directory; run `./run-test.py` in
143+
there to execute it. You can pass `--target` to execute the test for another target.
145144

146145
### Using a modified standard library
147146

@@ -287,3 +286,41 @@ https. Add the following to your `.gitconfig`:
287286
[url "git@github.com:"]
288287
pushInsteadOf = https://github.com/
289288
```
289+
290+
## Internal environment variables
291+
292+
The following environment variables are *internal* and must not be used by
293+
anyone but Miri itself. They are used to communicate between different Miri
294+
binaries, and as such worth documenting:
295+
296+
* `CARGO_EXTRA_FLAGS` is understood by `./miri` and passed to all host cargo invocations.
297+
* `MIRI_BE_RUSTC` can be set to `host` or `target`. It tells the Miri driver to
298+
actually not interpret the code but compile it like rustc would. With `target`, Miri sets
299+
some compiler flags to prepare the code for interpretation; with `host`, this is not done.
300+
This environment variable is useful to be sure that the compiled `rlib`s are compatible
301+
with Miri.
302+
* `MIRI_CALLED_FROM_SETUP` is set during the Miri sysroot build,
303+
which will re-invoke `cargo-miri` as the `rustc` to use for this build.
304+
* `MIRI_CALLED_FROM_RUSTDOC` when set to any value tells `cargo-miri` that it is
305+
running as a child process of `rustdoc`, which invokes it twice for each doc-test
306+
and requires special treatment, most notably a check-only build before interpretation.
307+
This is set by `cargo-miri` itself when running as a `rustdoc`-wrapper.
308+
* `MIRI_CWD` when set to any value tells the Miri driver to change to the given
309+
directory after loading all the source files, but before commencing
310+
interpretation. This is useful if the interpreted program wants a different
311+
working directory at run-time than at build-time.
312+
* `MIRI_LOCAL_CRATES` is set by `cargo-miri` to tell the Miri driver which
313+
crates should be given special treatment in diagnostics, in addition to the
314+
crate currently being compiled.
315+
* `MIRI_ORIG_RUSTDOC` is set and read by different phases of `cargo-miri` to remember the
316+
value of `RUSTDOC` from before it was overwritten.
317+
* `MIRI_REPLACE_LIBRS_IF_NOT_TEST` when set to any value enables a hack that helps bootstrap
318+
run the standard library tests in Miri.
319+
* `MIRI_TEST_TARGET` is set by `./miri test` (and `./x.py test miri`) to tell the test harness about
320+
the chosen target.
321+
* `MIRI_VERBOSE` when set to any value tells the various `cargo-miri` phases to
322+
perform verbose logging.
323+
* `MIRI_HOST_SYSROOT` is set by bootstrap to tell `cargo-miri` which sysroot to use for *host*
324+
operations.
325+
* `RUSTC_BLESS` is set by `./miri test` (and `./x.py test miri`) to indicate bless-mode to the test
326+
harness.

‎src/tools/miri/README.md

+38-95
Original file line numberDiff line numberDiff line change
@@ -1,39 +1,34 @@
11
# Miri
22

3-
An experimental interpreter for [Rust][rust]'s
4-
[mid-level intermediate representation][mir] (MIR). It can run binaries and
5-
test suites of cargo projects and detect certain classes of
6-
[undefined behavior](https://doc.rust-lang.org/reference/behavior-considered-undefined.html),
7-
for example:
3+
Miri is an [Undefined Behavior][reference-ub] detection tool for Rust. It can run binaries and test
4+
suites of cargo projects and detect unsafe code that fails to uphold its safety requirements. For
5+
instance:
86

97
* Out-of-bounds memory accesses and use-after-free
108
* Invalid use of uninitialized data
119
* Violation of intrinsic preconditions (an [`unreachable_unchecked`] being
1210
reached, calling [`copy_nonoverlapping`] with overlapping ranges, ...)
1311
* Not sufficiently aligned memory accesses and references
14-
* Violation of *some* basic type invariants (a `bool` that is not 0 or 1, for example,
12+
* Violation of basic type invariants (a `bool` that is not 0 or 1, for example,
1513
or an invalid enum discriminant)
1614
* **Experimental**: Violations of the [Stacked Borrows] rules governing aliasing
1715
for reference types
1816
* **Experimental**: Violations of the [Tree Borrows] aliasing rules, as an optional
1917
alternative to [Stacked Borrows]
20-
* **Experimental**: Data races
18+
* **Experimental**: Data races and emulation of weak memory effects, i.e.,
19+
atomic reads can return outdated values.
2120

2221
On top of that, Miri will also tell you about memory leaks: when there is memory
2322
still allocated at the end of the execution, and that memory is not reachable
2423
from a global `static`, Miri will raise an error.
2524

26-
Miri supports almost all Rust language features; in particular, unwinding and
27-
concurrency are properly supported (including some experimental emulation of
28-
weak memory effects, i.e., reads can return outdated values).
29-
3025
You can use Miri to emulate programs on other targets, e.g. to ensure that
3126
byte-level data manipulation works correctly both on little-endian and
3227
big-endian systems. See
3328
[cross-interpretation](#cross-interpretation-running-for-different-targets)
3429
below.
3530

36-
Miri has already discovered some [real-world bugs](#bugs-found-by-miri). If you
31+
Miri has already discovered many [real-world bugs](#bugs-found-by-miri). If you
3732
found a bug with Miri, we'd appreciate if you tell us and we'll add it to the
3833
list!
3934

@@ -45,33 +40,36 @@ clocks, are replaced by deterministic "fake" implementations. Set
4540
(In particular, the "fake" system RNG APIs make Miri **not suited for
4641
cryptographic use**! Do not generate keys using Miri.)
4742

48-
All that said, be aware that Miri will **not catch all cases of undefined
49-
behavior** in your program, and cannot run all programs:
43+
All that said, be aware that Miri does **not catch every violation of the Rust specification** in
44+
your program, not least because there is no such specification. Miri uses its own approximation of
45+
what is and is not Undefined Behavior in Rust. To the best of our knowledge, all Undefined Behavior
46+
that has the potential to affect a program's correctness *is* being detected by Miri (modulo
47+
[bugs][I-misses-ub]), but you should consult [the Reference][reference-ub] for the official
48+
definition of Undefined Behavior. Miri will be updated with the Rust compiler to protect against UB
49+
as it is understood by the current compiler, but it makes no promises about future versions of
50+
rustc.
5051

51-
* There are still plenty of open questions around the basic invariants for some
52-
types and when these invariants even have to hold. Miri tries to avoid false
53-
positives here, so if your program runs fine in Miri right now that is by no
54-
means a guarantee that it is UB-free when these questions get answered.
52+
Further caveats that Miri users should be aware of:
5553

56-
In particular, Miri does not check that references point to valid data.
5754
* If the program relies on unspecified details of how data is laid out, it will
5855
still run fine in Miri -- but might break (including causing UB) on different
59-
compiler versions or different platforms.
56+
compiler versions or different platforms. (You can use `-Zrandomize-layout`
57+
to detect some of these cases.)
6058
* Program execution is non-deterministic when it depends, for example, on where
6159
exactly in memory allocations end up, or on the exact interleaving of
6260
concurrent threads. Miri tests one of many possible executions of your
63-
program. You can alleviate this to some extent by running Miri with different
64-
values for `-Zmiri-seed`, but that will still by far not explore all possible
65-
executions.
61+
program, but it will miss bugs that only occur in a different possible execution.
62+
You can alleviate this to some extent by running Miri with different
63+
values for `-Zmiri-seed`, but that will still by far not explore all possible executions.
6664
* Miri runs the program as a platform-independent interpreter, so the program
6765
has no access to most platform-specific APIs or FFI. A few APIs have been
6866
implemented (such as printing to stdout, accessing environment variables, and
6967
basic file system access) but most have not: for example, Miri currently does
7068
not support networking. System API support varies between targets; if you run
7169
on Windows it is a good idea to use `--target x86_64-unknown-linux-gnu` to get
7270
better support.
73-
* Weak memory emulation may [produce weak behaviours](https://github.com/rust-lang/miri/issues/2301)
74-
unobservable by compiled programs running on real hardware when `SeqCst` fences are used, and it
71+
* Weak memory emulation may [produce weak behaviors](https://github.com/rust-lang/miri/issues/2301)
72+
when `SeqCst` fences are used that are not actually permitted by the Rust memory model, and it
7573
cannot produce all behaviors possibly observable on real hardware.
7674

7775
Moreover, Miri fundamentally cannot tell you whether your code is *sound*. [Soundness] is the property
@@ -87,6 +85,8 @@ coverage.
8785
[Stacked Borrows]: https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md
8886
[Tree Borrows]: https://perso.crans.org/vanille/treebor/
8987
[Soundness]: https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library
88+
[reference-ub]: https://doc.rust-lang.org/reference/behavior-considered-undefined.html
89+
[I-misses-ub]: https://github.com/rust-lang/miri/labels/I-misses-UB
9090

9191

9292
## Using Miri
@@ -97,14 +97,8 @@ Install Miri on Rust nightly via `rustup`:
9797
rustup +nightly component add miri
9898
```
9999

100-
If `rustup` says the `miri` component is unavailable, that's because not all
101-
nightly releases come with all tools. Check out
102-
[this website](https://rust-lang.github.io/rustup-components-history) to
103-
determine a nightly version that comes with Miri and install that using `rustup
104-
toolchain install nightly-YYYY-MM-DD`. Either way, all of the following commands
105-
assume the right toolchain is pinned via `rustup override set nightly` or
106-
`rustup override set nightly-YYYY-MM-DD`. (Alternatively, use `cargo
107-
+nightly`/`cargo +nightly-YYYY-MM-DD` for each of the following commands.)
100+
All the following commands assume the nightly toolchain is pinned via `rustup override set nightly`.
101+
Alternatively, use `cargo +nightly` for each of the following commands.
108102

109103
Now you can run your project in Miri:
110104

@@ -118,12 +112,12 @@ dependencies. It will ask you for confirmation before installing anything.
118112
example, `cargo miri test filter` only runs the tests containing `filter` in
119113
their name.
120114

121-
You can pass arguments to Miri via `MIRIFLAGS`. For example,
115+
You can pass [flags][miri-flags] to Miri via `MIRIFLAGS`. For example,
122116
`MIRIFLAGS="-Zmiri-disable-stacked-borrows" cargo miri run` runs the program
123117
without checking the aliasing of references.
124118

125119
When compiling code via `cargo miri`, the `cfg(miri)` config flag is set for code
126-
that will be interpret under Miri. You can use this to ignore test cases that fail
120+
that will be interpreted under Miri. You can use this to ignore test cases that fail
127121
under Miri because they do things Miri does not support:
128122

129123
```rust
@@ -159,10 +153,8 @@ endian-sensitive code.
159153

160154
### Running Miri on CI
161155

162-
To run Miri on CI, make sure that you handle the case where the latest nightly
163-
does not ship the Miri component because it currently does not build. `rustup
164-
toolchain install --component` knows how to handle this situation, so the
165-
following snippet should always work:
156+
When running Miri on CI, use the following snippet to install a nightly toolchain with the Miri
157+
component:
166158

167159
```sh
168160
rustup toolchain install nightly --component miri
@@ -227,7 +219,7 @@ degree documented below):
227219
- We have unofficial support (not maintained by the Miri team itself) for some further operating systems.
228220
- `freebsd`: **maintainer wanted**. Supports `std::env` and parts of `std::{thread, fs}`, but not `std::sync`.
229221
- `android`: **maintainer wanted**. Support very incomplete, but a basic "hello world" works.
230-
- `illumos`: maintained by @devnexen. Support very incomplete, but a basic "hello world" works.
222+
- `solaris` / `illumos`: maintained by @devnexen. Support very incomplete, but a basic "hello world" works.
231223
- `wasm`: **maintainer wanted**. Support very incomplete, not even standard output works, but an empty `main` function works.
232224
- For targets on other operating systems, Miri might fail before even reaching the `main` function.
233225

@@ -273,25 +265,12 @@ To get a backtrace, you need to disable isolation
273265
RUST_BACKTRACE=1 MIRIFLAGS="-Zmiri-disable-isolation" cargo miri test
274266
```
275267

276-
#### "found possibly newer version of crate `std` which `<dependency>` depends on"
277-
278-
Your build directory may contain artifacts from an earlier build that have/have
279-
not been built for Miri. Run `cargo clean` before switching from non-Miri to
280-
Miri builds and vice-versa.
281-
282268
#### "found crate `std` compiled by an incompatible version of rustc"
283269

284270
You may be running `cargo miri` with a different compiler version than the one
285271
used to build the custom libstd that Miri uses, and Miri failed to detect that.
286272
Try running `cargo miri clean`.
287273

288-
#### "no mir for `std::rt::lang_start_internal`"
289-
290-
This means the sysroot you are using was not compiled with Miri in mind. This
291-
should never happen when you use `cargo miri` because that takes care of setting
292-
up the sysroot. If you are using `miri` (the Miri driver) directly, see the
293-
[contributors' guide](CONTRIBUTING.md) for how to use `./miri` to best do that.
294-
295274

296275
## Miri `-Z` flags and environment variables
297276
[miri-flags]: #miri--z-flags-and-environment-variables
@@ -395,17 +374,17 @@ to Miri failing to detect cases of undefined behavior in a program.
395374
this flag is **unsound**.
396375
* `-Zmiri-disable-weak-memory-emulation` disables the emulation of some C++11 weak
397376
memory effects.
398-
* `-Zmiri-extern-so-file=<path to a shared object file>` is an experimental flag for providing support
399-
for FFI calls. Functions not provided by that file are still executed via the usual Miri shims.
400-
**WARNING**: If an invalid/incorrect `.so` file is specified, this can cause undefined behaviour in Miri itself!
401-
And of course, Miri cannot do any checks on the actions taken by the external code.
377+
* `-Zmiri-native-lib=<path to a shared object file>` is an experimental flag for providing support
378+
for calling native functions from inside the interpreter via FFI. Functions not provided by that
379+
file are still executed via the usual Miri shims.
380+
**WARNING**: If an invalid/incorrect `.so` file is specified, this can cause Undefined Behavior in Miri itself!
381+
And of course, Miri cannot do any checks on the actions taken by the native code.
402382
Note that Miri has its own handling of file descriptors, so if you want to replace *some* functions
403383
working on file descriptors, you will have to replace *all* of them, or the two kinds of
404384
file descriptors will be mixed up.
405385
This is **work in progress**; currently, only integer arguments and return values are
406386
supported (and no, pointer/integer casts to work around this limitation will not work;
407-
they will fail horribly). It also only works on unix hosts for now.
408-
Follow [the discussion on supporting other types](https://github.com/rust-lang/miri/issues/2365).
387+
they will fail horribly). It also only works on Linux hosts for now.
409388
* `-Zmiri-measureme=<name>` enables `measureme` profiling for the interpreted program.
410389
This can be used to find which parts of your program are executing slowly under Miri.
411390
The profile is written out to a file inside a directory called `<name>`, and can be processed
@@ -484,50 +463,14 @@ by all intended entry points, i.e. `cargo miri` and `./miri {test,run}`):
484463
* `MIRI_SYSROOT` indicates the sysroot to use. When using `cargo miri`, this skips the automatic
485464
setup -- only set this if you do not want to use the automatically created sysroot. When invoking
486465
`cargo miri setup`, this indicates where the sysroot will be put.
487-
* `MIRI_TEST_TARGET` (recognized by `./miri {test,run}`) indicates which target
488-
architecture to test against. `miri` and `cargo miri` accept the `--target` flag for the same
489-
purpose.
490466
* `MIRI_TEST_THREADS` (recognized by `./miri test`): set the number of threads to use for running tests.
491467
By default, the number of cores is used.
492468
* `MIRI_NO_STD` makes sure that the target's sysroot is built without libstd. This allows testing
493469
and running no_std programs. (Miri has a heuristic to detect no-std targets based on the target
494470
name; this environment variable is only needed when that heuristic fails.)
495-
* `RUSTC_BLESS` (recognized by `./miri test` and `cargo-miri-test/run-test.py`): overwrite all
496-
`stderr` and `stdout` files instead of checking whether the output matches.
497471
* `MIRI_SKIP_UI_CHECKS` (recognized by `./miri test`): don't check whether the
498472
`stderr` or `stdout` files match the actual output.
499473

500-
The following environment variables are *internal* and must not be used by
501-
anyone but Miri itself. They are used to communicate between different Miri
502-
binaries, and as such worth documenting:
503-
504-
* `MIRI_BE_RUSTC` can be set to `host` or `target`. It tells the Miri driver to
505-
actually not interpret the code but compile it like rustc would. With `target`, Miri sets
506-
some compiler flags to prepare the code for interpretation; with `host`, this is not done.
507-
This environment variable is useful to be sure that the compiled `rlib`s are compatible
508-
with Miri.
509-
* `MIRI_CALLED_FROM_SETUP` is set during the Miri sysroot build,
510-
which will re-invoke `cargo-miri` as the `rustc` to use for this build.
511-
* `MIRI_CALLED_FROM_RUSTDOC` when set to any value tells `cargo-miri` that it is
512-
running as a child process of `rustdoc`, which invokes it twice for each doc-test
513-
and requires special treatment, most notably a check-only build before interpretation.
514-
This is set by `cargo-miri` itself when running as a `rustdoc`-wrapper.
515-
* `MIRI_CWD` when set to any value tells the Miri driver to change to the given
516-
directory after loading all the source files, but before commencing
517-
interpretation. This is useful if the interpreted program wants a different
518-
working directory at run-time than at build-time.
519-
* `MIRI_LOCAL_CRATES` is set by `cargo-miri` to tell the Miri driver which
520-
crates should be given special treatment in diagnostics, in addition to the
521-
crate currently being compiled.
522-
* `MIRI_ORIG_RUSTDOC` is set and read by different phases of `cargo-miri` to remember the
523-
value of `RUSTDOC` from before it was overwritten.
524-
* `MIRI_REPLACE_LIBRS_IF_NOT_TEST` when set to any value enables a hack that helps bootstrap
525-
run the standard library tests in Miri.
526-
* `MIRI_VERBOSE` when set to any value tells the various `cargo-miri` phases to
527-
perform verbose logging.
528-
* `MIRI_HOST_SYSROOT` is set by bootstrap to tell `cargo-miri` which sysroot to use for *host*
529-
operations.
530-
531474
[testing-miri]: CONTRIBUTING.md#testing-the-miri-driver
532475

533476
## Miri `extern` functions

0 commit comments

Comments
 (0)