Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fully remove submodule handling from bootstrap.py #97513

Merged
merged 4 commits into from
Jun 25, 2022
Merged

Fully remove submodule handling from bootstrap.py #97513

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9cde0f7
Fully remove submodule handling from bootstrap.py
jyn514 May 29, 2022
85c87f6
Add bootstrap to tidy check
jyn514 May 29, 2022
345eb14
Remove vendoring support when building from git sources
jyn514 Jun 22, 2022
81482e6
Remove individual crate checks for bootstrap in tidy
jyn514 Jun 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Remove vendoring support when building from git sources 
This is difficult to support without submodule handling in bootstrap.py, because cargo will refuse
to vendor sources unless it knows the Cargo.toml files of all tools in tree. Moving vendor support
to rustbuild means that rustbuild will be built without vendoring.

Rather than trying to solve this, just remove support altogether and require
people to use `rustc-src` if they want vendoring (or run `cargo vendor` manually).
  • Loading branch information
@jyn514
jyn514 committed Jun 22, 2022
commit 345eb14f6c841cd38e76a5b0bbf99e1b94a90d40
3 changes: 2 additions & 1 deletion src/bootstrap/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [Changes since the last major version]

- Vendoring is no longer done automatically when building from git sources. To use vendoring, run `cargo vendor` manually, or use the pre-vendored `rustc-src` tarball.
- `llvm-libunwind` now accepts `in-tree` (formerly true), `system` or `no` (formerly false) [#77703](https://github.com/rust-lang/rust/pull/77703)
- The options `infodir`, `localstatedir`, and `gpg-password-file` are no longer allowed in config.toml. Previously, they were ignored without warning. Note that `infodir` and `localstatedir` are still accepted by `./configure`, with a warning. [#82451](https://github.com/rust-lang/rust/pull/82451)
- Add options for enabling overflow checks, one for std (`overflow-checks-std`) and one for everything else (`overflow-checks`). Both default to false.
- Change the names for `dist` commands to match the component they generate. [#90684](https://github.com/rust-lang/rust/pull/90684)
- The `build.fast-submodules` option has been removed. Fast submodule checkouts are enabled unconditionally. Automatic submodule handling can still be disabled with `build.submodules = false`.

Expand All @@ -19,6 +19,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- The default bootstrap profiles are now located at `bootstrap/defaults/config.$PROFILE.toml` (previously they were located at `bootstrap/defaults/config.toml.$PROFILE`) [#77558](https://github.com/rust-lang/rust/pull/77558)
- If you have Rust already installed, `x.py` will now infer the host target
from the default rust toolchain. [#78513](https://github.com/rust-lang/rust/pull/78513)
- Add options for enabling overflow checks, one for std (`overflow-checks-std`) and one for everything else (`overflow-checks`). Both default to false.


## [Version 2] - 2020-09-25
Expand Down
58 changes: 17 additions & 41 deletions src/bootstrap/bootstrap.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -822,55 +822,32 @@ def set_dist_environment(self, url):

def check_vendored_status(self):
"""Check that vendoring is configured properly"""
vendor_dir = os.path.join(self.rust_root, 'vendor')
if 'SUDO_USER' in os.environ and not self.use_vendored_sources:
if os.getuid() == 0:
self.use_vendored_sources = True
print('info: looks like you\'re trying to run this command as root')
print(' and so in order to preserve your $HOME this will now')
print(' use vendored sources by default.')
if not os.path.exists(vendor_dir):
print('error: vendoring required, but vendor directory does not exist.')
print(' Run `cargo vendor` without sudo to initialize the '
'vendor directory.')
raise Exception("{} not found".format(vendor_dir))

cargo_dir = os.path.join(self.rust_root, '.cargo')
if self.use_vendored_sources:
config = ("[source.crates-io]\n"
"replace-with = 'vendored-sources'\n"
"registry = 'https://example.com'\n"
"\n"
"[source.vendored-sources]\n"
"directory = '{}/vendor'\n"
.format(self.rust_root))
if not os.path.exists('.cargo'):
os.makedirs('.cargo')
with output('.cargo/config') as cargo_config:
cargo_config.write(config)
else:
print('info: using vendored source, but .cargo/config is already present.')
print(' Reusing the current configuration file. But you may want to '
'configure vendoring like this:')
print(config)
vendor_dir = os.path.join(self.rust_root, 'vendor')
if not os.path.exists(vendor_dir):
sync_dirs = "--sync ./src/tools/rust-analyzer/Cargo.toml " \
"--sync ./compiler/rustc_codegen_cranelift/Cargo.toml " \
"--sync ./src/bootstrap/Cargo.toml "
print('error: vendoring required, but vendor directory does not exist.')
print(' Run `cargo vendor {}` to initialize the '
'vendor directory.'.format(sync_dirs))
print('Alternatively, use the pre-vendored `rustc-src` dist component.')
raise Exception("{} not found".format(vendor_dir))

if not os.path.exists(cargo_dir):
print('error: vendoring required, but .cargo/config does not exist.')
raise Exception("{} not found".format(cargo_dir))
else:
if os.path.exists('.cargo'):
shutil.rmtree('.cargo')

def ensure_vendored(self):
"""Ensure that the vendored sources are available if needed"""
vendor_dir = os.path.join(self.rust_root, 'vendor')
# Note that this does not handle updating the vendored dependencies if
# the rust git repository is updated. Normal development usually does
# not use vendoring, so hopefully this isn't too much of a problem.
if self.use_vendored_sources and not os.path.exists(vendor_dir):
run([
self.cargo(),
"vendor",
"--sync=./src/bootstrap/Cargo.toml",
"--sync=./src/tools/rust-analyzer/Cargo.toml",
"--sync=./compiler/rustc_codegen_cranelift/Cargo.toml",
], verbose=self.verbose, cwd=self.rust_root)

if os.path.exists(cargo_dir):
shutil.rmtree(cargo_dir)

def bootstrap(help_triggered):
"""Configure, fetch, build and run the initial bootstrap"""
Expand Down Expand Up @@ -953,7 +930,6 @@ def bootstrap(help_triggered):
# Fetch/build the bootstrap
build.download_toolchain()
sys.stdout.flush()
build.ensure_vendored()
build.build_bootstrap()
sys.stdout.flush()

Expand Down
13 changes: 12 additions & 1 deletion src/bootstrap/dist.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -894,8 +894,19 @@ impl Step for PlainSourceTarball {
.arg(builder.src.join("./src/tools/rust-analyzer/Cargo.toml"))
.arg("--sync")
.arg(builder.src.join("./compiler/rustc_codegen_cranelift/Cargo.toml"))
.arg("--sync")
.arg(builder.src.join("./src/bootstrap/Cargo.toml"))
.current_dir(&plain_dst_src);
builder.run(&mut cmd);

let config = if !builder.config.dry_run {
t!(String::from_utf8(t!(cmd.output()).stdout))
} else {
String::new()
};

let cargo_config_dir = plain_dst_src.join(".cargo");
builder.create_dir(&cargo_config_dir);
builder.create(&cargo_config_dir.join("config.toml"), &config);
}

tarball.bare()
Expand Down