clang assumes zero-extension of 8-bit arguments in x86, causing interop issues with gcc

@topperc


Bugzilla Link	44228
Version	trunk
OS	Linux
Attachments	test-case
CC	@topperc,@davezarzycki,@froydnj,@jrmuizel,@josephcsible,@jdm,@jyknight,@RKSimon,@zygoloid,@rjmccall,@tstellar

Extended Description

When receiving 8-bit-wide arguments in extern function, clang seems to assume the argument has been zero-extended by the caller.

According to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92821#c2:

I believe it is a LLVM bug.
At least, reading https://github.com/hjl-tools/x86-psABI/wiki/x86-64-psABI-1.0.pdf, I can't find in the Parameter Passing section anything that would say that arguments smaller than 64-bit are passed sign or zero extended to 64-bit like some other psABIs require. The only related thing is
"When a value of type _Bool is returned or passed in a register or on the stack, bit 0 contains the truth value and bits 1 to 7 shall be zero."
with a footnote:
"Other bits are left unspecified, hence the consumer side of those values can rely on it being 0 or 1 when truncated to 8 bit."
which says that _Bool has only significant low 8 bits and the rest is unspecified.

https://godbolt.org/z/BNHxEY has a comparison of clang and gcc output for the attached test-case. GCC correctly does an 8-bit load, disregarding the rest of the bits in the register.

This causes real problems when gcc-built functions call into llvm-built functions. See https://bugzilla.mozilla.org/show_bug.cgi?id=1600735 for an example that happens on Firefox. GCC may not always sign-extend in the caller.

In the Firefox case the LLVM-built function is Rust code, but per the above godbolt link it also seems to reproduce with C / C++.

I believe this may be a dupe / related to bug 12207, though it seems like a somewhat serious issue.

I sent a tentative patch for this in https://reviews.llvm.org/D71178.

I think it'd do the right thing, but I still have 10 or so tests to update.

Ugh....

LLVM has been using the caller-extends-to-32bit ABI for...just about ever. And, except perhaps for this particular bug, GCC also has been generating calls compliant with that. And per https://gcc.gnu.org/PR46942 wanted to depend on it as well -- only didn't due to (at the time) difficulties in codegen.

There also appeared to be overall a desire to change the ABI document to state that....but for some reason never happened (with no further followup AFAIK), which is really unfortunate.

So, I don't agree this is an LLVM bug, at least not without reopening the discussion on the x86-64 ABI list and coming to some final conclusion on how to update the document to clarify matters.

I think:

The ABI document should be actually modified to record that caller-extends is the ABI. (Because that is de-facto the ABI already.)
Any odd corner-case where the value is not extended by callers should be fixed. Sounds like the one known case in GCC has indeed already been fixed, but it's not clear if that was intentional.
If the outcome of a discussion on the ABI list is that, despite historical precedent, no zero/sign extension to 32-its is required on the caller side, LLVM should still continue to extend there, for compatibility with older versions of itself.

Ugh....

LLVM has been using the caller-extends-to-32bit ABI for...just about ever.
And, except perhaps for this particular bug, GCC also has been generating
calls compliant with that. And per https://gcc.gnu.org/#46942 wanted to
depend on it as well -- only didn't due to (at the time) difficulties in
codegen.

There also appeared to be overall a desire to change the ABI document to
state that....but for some reason never happened (with no further followup
AFAIK), which is really unfortunate.

So, I don't agree this is an LLVM bug, at least not without reopening the
discussion on the x86-64 ABI list and coming to some final conclusion on how
to update the document to clarify matters.

I think:

The ABI document should be actually modified to record that
caller-extends is the ABI. (Because that is de-facto the ABI already.)

Any odd corner-case where the value is not extended by callers should
be fixed. Sounds like the one known case in GCC has indeed already been
fixed, but it's not clear if that was intentional.

If the outcome of a discussion on the ABI list is that, despite
historical precedent, no zero/sign extension to 32-its is required on the
caller side, LLVM should still continue to extend there, for compatibility
with older versions of itself.

That all seems reasonable, thank you.

Since ABI document doesn't define the behavior, we can't assume compiler other than clang would sign/zero extend parameter in caller. We should be compatible to other compiler as much as possible. So I think we should remove the sign/zero extend in callee. Removing the assumption (caller has sign/zero extend) in callee would make Clang be compatible with previous clang compiler, gcc and ICC. I didn't see any side effect on doing this.

And yet, de-facto, other compilers DO sign/zero extend in caller, and therefore DO interop with Clang. It seems silly to require extension on both sides of the call (though, of course, GCC has been doing that).

However ICC doesn't sign/zero extend in caller. See https://gcc.godbolt.org/z/8aaPP33cG. So if the caller is built by ICC and callee is built by Clang, the issue happen. We can't assume all compiler do sign/zero extend in caller.

Ah, yes; that's an unfortunate new piece of information here. :(

Also, GCC doesn't guarantee the sign extension.

It usually does, due to integer promotion rules, but it might not if you're passing e.g. an enum class or so. The Mozilla bug referenced at the top of the issue was caused by that, and it was GCC code calling into Rust (LLVM) code.

Since it's not clear on the bug trail (discussions happened elsewhere) --

The consensus from discussions on the ABI list and elsewhere is that Clang is indeed incorrect per the x86 psABI specification. I think that's unfortunate (and it also would've been nice if the spec had been more clearly written from the get-go) but being that as it may: the consensus is that x86 psABI specifies that no zero/sign-extension is required for callers passing 8/16-bit values, so this is a Clang bug that needs to be fixed.

The only issue for Clang is that it should fix this without breaking compatibility with old versions of itself. Thus, it must stop assuming values have been extended in the callee, but continue to (possibly-redundantly) extend in the caller.

I agree that the spec was perhaps unclear for char, but _Bool is also impacted: #12579 (comment)

Got bitten by this today: https://play.rust-lang.org/?version=stable&mode=release&edition=2024&gist=58b20113756fdb4dfc83f434f4951a63

#[unsafe(no_mangle)]
pub extern "C" fn index(b: u8) -> usize {
    b as usize
}

If the index function is used from GCC generated code

index:
	movl	%edi, %eax
	retq

then it becomes a index out of bound read because the upper bits may contain garbage data.

Since it's not clear on the bug trail (discussions happened elsewhere) --

So it doesn't get lost, I believe this was referring to https://groups.google.com/g/x86-64-abi/c/h7FFh30oS3s/m/m8y6vrdCAQAJ

So, I don't agree this is an LLVM bug, at least not without reopening the discussion on the x86-64 ABI list and coming to some final conclusion on how to update the document to clarify matters.

I think it's definitely an LLVM bug. The contract LLVM has to implement is defined by the psABI, and that contract does not grant LLVM any license to assume that things are zero-extended.

The fact that some other compilers happen to zero-extend arguments in many cases doesn't change anything about it. We can't conclude a guarantee from what they happen to do, unless they explicitly document it as a guarantee.

Is this something the frontend can work around by emitting different attributes, or does it need changes in the backend?

It also seems like meanwhile, the psABI has been clarified to say explicitly that the higher bits are unspecified.

I think it's definitely an LLVM bug.

Yes, you quoted an old comment...as I wrote on Feb 26, 2024, everyone has agreed this is definitely a bug in Clang since the discussions in 2024 (which then resulted in the psABI document being clarified).

Is this something the frontend can work around by emitting different attributes, or does it need changes in the backend?

On x86-64, we currently lower a "short" parameter to "i16 signext". This both tells callers to sign-extend their argument to 32-bits, and tells callees that the parameter is guaranteed to have been sign-extended.

We could easily use "i16" for the parameter instead, which implies the upper bits are unknown in the callee, which is the correct behavior. BUT the problem is that if we do that, callers will stop sign-extending to 32-bits as well, and that will break things if a caller built with new-clang calls a callee built with old-Clang.

So: we need to make some change which will both ensure that callers continue to treat such a parameter as requiring sign-extension, but not assume it has been extended when compiling the callee.

IIRC, https://reviews.llvm.org/D124435 is the last try in fixing it.

@jyknight

I think it's definitely an LLVM bug.

Yes, you quoted an old comment...as I wrote on Feb 26, 2024, everyone has agreed this is definitely a bug in Clang since the discussions in 2024 (which then resulted in the psABI document being clarified).

@jyknight Just to clarify, Clang's behavior with _Bool is incorrect as well. I have not seen anyone claiming the old psABI specification was unclear. It explicitly said the upper bits that Clang relies on being zero are unspecified.

Navigation Menu

Repository navigation

clang assumes zero-extension of 8-bit arguments in x86, causing interop issues with gcc #43573

Extended Description

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

clang assumes zero-extension of 8-bit arguments in x86, causing interop issues with gcc #43573

Description

Extended Description

Activity

emilio commented on Dec 5, 2019

emilio commented on Dec 9, 2019

jyknight commented on Mar 17, 2020

rjmccall commented on Mar 18, 2020

LuoYuanke commented on Mar 31, 2022

jyknight commented on Mar 31, 2022

LuoYuanke commented on Mar 31, 2022

jyknight commented on Mar 31, 2022

emilio commented on Mar 31, 2022

jyknight commented on Feb 26, 2024

fweimer-rh commented on Mar 10, 2026

kanru commented on Apr 10, 2026

tgross35 commented on Jul 14, 2026

RalfJung commented on Jul 23, 2026

RalfJung commented on Jul 23, 2026

jyknight commented on Jul 23, 2026

phoebewang commented on Jul 24, 2026

fweimer-rh commented on Jul 24, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions