Skip to content

Conversation

@geofft
Copy link
Collaborator

@geofft geofft commented Oct 21, 2025

Python uses siphash (siphash13 in 3.11+, siphash24 on older versions) as the default internal hashing algorithm, but only on architectures that support misaligned accesses, i.e., reads/writes of integers from a memory address that is not a round multiple of the integer size. On other architectures it uses fnv, which is not supported by Numba and raises a warning. The distinction between architectures is done by a configure-time code execution check, which is not supported on our cross builds, including on our x86_64_vN microarchitecture builds (see #599), so Python defaults to assuming it is not supported.

Hard-code a list of platforms that are known to support misaligned accesses just fine. Credit to
https://blog.vitlabuda.cz/2025/01/22/unaligned-memory-access-on-various-cpu-architectures.html for pointing out that the Linux kernel has this pretty well documented in Kconfig.

Note that loongarch and riscv have optional support for misaligned access, and it's quite possible that the hardware that people actually use have support for them (or that we are targeting a limited hardware profile anyway for some reason that implies support for misaligned access). I've left them out for now but we can add them later.

Fixes #683.

Python uses siphash (siphash13 in 3.11+, siphash24 on older versions) as
the default internal hashing algorithm, but only on architectures that
support misaligned accesses, i.e., reads/writes of integers from a
memory address that is not a round multiple of the integer size. On
other architectures it uses fnv, which is not supported by Numba and raises
a warning. The distinction between architectures is done by a
configure-time code execution check, which is not supported on our cross
builds, including on our x86_64_vN microarchitecture builds (see #599),
so Python defaults to assuming it is not supported.

Hard-code a list of platforms that are known to support misaligned
accesses just fine. Credit to
https://blog.vitlabuda.cz/2025/01/22/unaligned-memory-access-on-various-cpu-architectures.html
for pointing out that the Linux kernel has this pretty well documented
in Kconfig.

Note that loongarch and riscv have optional support for misaligned
access, and it's quite possible that the hardware that people actually
use have support for them (or that we are targeting a limited hardware
profile anyway for some reason that implies support for misaligned
access). I've left them out for now but we can add them later.

Fixes #683.
Copy link
Member

@zanieb zanieb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<3

@geofft geofft merged commit 5dcb8cc into main Oct 21, 2025
569 checks passed
@geofft geofft deleted the geofft/aligned-required-cross branch October 21, 2025 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Missing siphash24 support generates warnings in numba

2 participants