You can use e.g. --with-lg-page=16 to cross-compile to a target that uses 64 KiB pages. jemalloc used to check page size during bootstrapping, but this had two shortcomings. First, it was difficult to avoid recursive allocation on all platforms, and second, making the page size dynamic prevented some compiler optimizations. IIRC Solaris is the only platform I could find on which it was conceivable for a single system to use multiple page sizes (thus warranting a run-time check), but even there it was not a practical concern.

I think in this case, Rust would like to have binaries that run everywhere, not subject to the configuration whims of different distros. So --with-lg-page=16 just makes a different non-portable binary, rather than something universal.

Is it feasible to run jemalloc compiled with the 64k page size even on systems that are only 4k? I guess these madvise calls would be fine, but general mmap will probably be surprising if it returns addresses aligned less than expected. And Mozilla bug 1091515 showed real problems with munmap (on mozjemalloc).

I'm not surprised that dynamic page size reduces compiler optimizations. Perhaps some of that can be mitigated with different expressions, e.g. instead of page mask, use a shr+shl combo to strip bits. But I'm sure that's a lot of code to deal with.

FWIW a single system could have different page sizes with different kernels -- e.g. a kernel developer who normally runs the distro kernel, but sometimes reboots into their own build. Obviously they would not want to swap out all userspace each time. But maybe mixed pages are impractical for other reasons. There may be other programs with static page sizes too, and that's normally just fine in a controlled distro.

Earlier this year I tried making it possible to specify a larger page size than the true system page size (245ae60), but I had to revert it (05a9e4a) because it could cause VM map fragmentation.

What about --with-lg-page-sizes? That sounds like it's supposed to support a hybrid configuration.

--with-lg-page-sizes is only useful when generating headers that will be imported into another build system that doesn't run configure again, e.g. in FreeBSD's libc, which in turn can be built for a variety of architectures.

More than a cross-distro problem, it can cause problems in a single distro, where the user switches the page size it their kernel.

Now, something that /could/ work, is if one configured --with-lg-pages for the biggest you can expect a kernel to use. The problem is that it then brakes chunk_alloc_mmap_slow when the runtime page size is actually smaller, and you end up with chunks actually smaller than the chunk size. Fun ensues.

How expensive would it be (in code bloat, etc) to have multiple copies of the code compiled for different constants (to still get the benefit of compiler optimizations which are aware of the page size)? So you basically compile three different versions of the code - the 4K page version, the 64K page version, and, if you're being really careful, the X page version (where X must be determined at runtime). Then, at init time when a process starts, you decide which one you're going to use.

Page size is baked into various data structures at compile time, so we would have to make substantial changes to directly support multiple page sizes at run time. However, it is possible to support a system page size smaller than the page size that is baked in. #769 implements such support.

Nice!

it is actively choosing to sabotage the ARM64 Linux ecosystem

@marcan IMO this comment is abusive. As much as I like your work (I even sponsor you), I don't think it's OK to insinuate malevolence.

As far as I understand, on ARM, assuming the compile-time page size is equivalent to the runtime one is exactly as non-portable as doing -march=native.

That is to say, in any scenario where a non-64K page size is hard-coded, architecturally speaking, -march=native is completely safe to add. So here's a free potential perf boost to add in the default jemalloc aarch64 linux configuration that does not add any additional portability concerns - compile with -march=native :) _{(might be slightly less trivial if you wanted nice error messages on mismatched compile-time vs runtime arch; and on aarch64 the difference between native and baseline isn't that large; but this is for making a point rather than being a serious suggestion ¯\_(ツ)_/¯)}

I might not quite call this active sabotage, but it's a pretty simple fact that essentially doing -march=native on a single architecture while not having equivalent behavior on others is a rather unusual stance for the default configuration.

I can understand the compiler optimization. But the pagesize is always a pow2 so you just do the masking for div/mod manually as the compiler would. (Non pow2 numbers need an additional mad AFAIK and for more info on this see libdivide) I don’t think it’s malicious just a bit too much focus on performance got in the way of practicality on ARM.

Linux has now boot time dynamic page size switch.

That makes page size a runtime selection, not something that is fixed, even for one OS distribution.