Conversation
Closure Trampoline Security Issue
=================================
Currently, the trampoline code used in libffi is not statically defined in
a source file (except for MACH). The trampoline is either pre-defined
machine code in a data buffer. Or, it is generated at runtime. In order to
execute a trampoline, it needs to be placed in a page with executable
permissions.
Executable data pages are attack surfaces for attackers who may try to
inject their own code into the page and contrive to have it executed. The
security settings in a system may prevent various tricks used in user land
to write code into a page and to have it executed somehow. On such systems,
libffi trampolines would not be able to run.
Static Trampoline
=================
To solve this problem, the trampoline code needs to be defined statically
in a source file, compiled and placed in the text segment so it can be
mapped and executed naturally without any tricks. However, the trampoline
needs to be able to access the closure pointer at runtime.
PC-relative data referencing
============================
The solution implemented in this patch set uses PC-relative data references.
The trampoline is mapped in a code page. Adjacent to the code page, a data
page is mapped that contains the parameters of the trampoline:
- the closure pointer
- pointer to the ABI handler to jump to
The trampoline code uses an offset relative to its current PC to access its
data.
Some architectures support PC-relative data references in the ISA itself.
E.g., X64 supports RIP-relative references. For others, the PC has to
somehow be loaded into a general purpose register to do PC-relative data
referencing. To do this, we need to define a get_pc() kind of function and
call it to load the PC in a desired register.
There are two cases:
1. The call instruction pushes the return address on the stack.
In this case, get_pc() will extract the return address from the stack
and load it in the desired register and return.
2. The call instruction stores the return address in a designated register.
In this case, get_pc() will copy the return address to the desired
register and return.
Either way, the PC next to the call instruction is obtained.
Scratch register
================
In order to do its job, the trampoline code would need to use a scratch
register. Depending on the ABI, there may not be a register available for
scratch. This problem needs to be solved so that all ABIs will work.
The trampoline will save two values on the stack:
- the closure pointer
- the original value of the scratch register
This is what the stack will look like:
sp before trampoline ------> --------------------
| closure pointer |
--------------------
| scratch register |
sp after trampoline -------> --------------------
The ABI handler can do the following as needed by the ABI:
- the closure pointer can be loaded in a desired register
- the scratch register can be restored to its original value
- the stack pointer can be restored to its original value
(the value when the trampoline was invoked)
To do this, I have defined prolog code for each ABI handler. The legacy
trampoline jumps to the ABI handler directly. But the static trampoline
defined in this patch jumps tp the prolog code which performs the above
actions before jumping to the ABI handler.
Trampoline Table
================
In order to reduce the trampoline memory footprint, the trampoline code
would be defined as a code array in the text segment. This array would be
mapped into the address space of the caller. The mapping would, therefore,
contain a trampoline table.
Adjacent to the trampoline table mapping, there will be a data mapping that
contains a parameter table, one parameter block for each trampoline. The
parameter block will contain:
- a pointer to the closure
- a pointer to the ABI handler
The static trampoline code would finally look like this:
- Make space on the stack for the closure and the scratch register
by moving the stack pointer down
- Store the original value of the scratch register on the stack
- Using PC-relative reference, get the closure pointer
- Store the closure pointer on the stack
- Using PC-relative reference, get the ABI handler pointer
- Jump to the ABI handler
Mapping size
============
The size of the code mapping that contains the trampoline table needs to be
determined on a per architecture basis. If a particular architecture
supports multiple base page sizes, then the largest supported base page size
needs to be chosen. E.g., we choose 16K for ARM64.
Trampoline allocation and free
==============================
Static trampolines are allocated in ffi_closure_alloc() and freed in
ffi_closure_free().
Normally, applications use these functions. But there are some cases out
there where the user of libffi allocates and manages its own closure
memory. In such cases, static trampolines cannot be used. These will
fall back to using legacy trampolines. The user has to make sure that
the memory is executable.
ffi_closure structure
=====================
I did not want to make any changes to the size of the closure structure for
this feature to guarantee compatibility. But the opaque static trampoline
handle needs to be stored in the closure. I have defined it as follows:
- char tramp[FFI_TRAMPOLINE_SIZE];
+ union {
+ char tramp[FFI_TRAMPOLINE_SIZE];
+ void *ftramp;
+ };
If static trampolines are used, then tramp[] is not needed to store a
dynamic trampoline. That space can be reused to store the handle. Hence,
the union.
Architecture Support
====================
Support has been added for x64, i386, aarch64 and arm. Support for other
architectures can be added very easily in the future.
OS Support
==========
Support has been added for Linux. Support for other OSes can be added very
easily.
Signed-off-by: Madhavan T. Venkataraman <[email protected]>
- Define the arch-specific initialization function ffi_tramp_arch () that returns trampoline size information to common code. - Define the trampoline code mapping and data mapping sizes. - Define the trampoline code table statically. Define two tables, actually, one with CET and one without. - Introduce a tiny prolog for each ABI handling function. The ABI handlers addressed are: - ffi_closure_unix64 - ffi_closure_unix64_sse - ffi_closure_win64 The prolog functions are called: - ffi_closure_unix64_alt - ffi_closure_unix64_sse_alt - ffi_closure_win64_alt The legacy trampoline jumps to the ABI handler. The static trampoline jumps to the prolog function. The prolog function uses the information provided by the static trampoline, sets things up for the ABI handler and then jumps to the ABI handler. - Call ffi_tramp_set_parms () in ffi_prep_closure_loc () to initialize static trampoline parameters. Signed-off-by: Madhavan T. Venkataraman <[email protected]>
- Define the arch-specific initialization function ffi_tramp_arch () that returns trampoline size information to common code. - Define the trampoline code table statically. Define two tables, actually, one with CET and one without. - Define the trampoline code table statically. - Introduce a tiny prolog for each ABI handling function. The ABI handlers addressed are: - ffi_closure_i386 - ffi_closure_STDCALL - ffi_closure_REGISTER The prolog functions are called: - ffi_closure_i386_alt - ffi_closure_STDCALL_alt - ffi_closure_REGISTER_alt The legacy trampoline jumps to the ABI handler. The static trampoline jumps to the prolog function. The prolog function uses the information provided by the static trampoline, sets things up for the ABI handler and then jumps to the ABI handler. - Call ffi_tramp_set_parms () in ffi_prep_closure_loc () to initialize static trampoline parameters. Signed-off-by: Madhavan T. Venkataraman <[email protected]>
- Define the arch-specific initialization function ffi_tramp_arch () that returns trampoline size information to common code. - Define the trampoline code mapping and data mapping sizes. - Define the trampoline code table statically. - Introduce a tiny prolog for each ABI handling function. The ABI handlers addressed are: - ffi_closure_SYSV - ffi_closure_SYSV_V The prolog functions are called: - ffi_closure_SYSV_alt - ffi_closure_SYSV_V_alt The legacy trampoline jumps to the ABI handler. The static trampoline jumps to the prolog function. The prolog function uses the information provided by the static trampoline, sets things up for the ABI handler and then jumps to the ABI handler. - Call ffi_tramp_set_parms () in ffi_prep_closure_loc () to initialize static trampoline parameters. Signed-off-by: Madhavan T. Venkataraman <[email protected]>
- Define the arch-specific initialization function ffi_tramp_arch () that returns trampoline size information to common code. - Define the trampoline code mapping and data mapping sizes. - Define the trampoline code table statically. - Introduce a tiny prolog for each ABI handling function. The ABI handlers addressed are: - ffi_closure_SYSV - ffi_closure_VFP The prolog functions are called: - ffi_closure_SYSV_alt - ffi_closure_VFP_alt The legacy trampoline jumps to the ABI handler. The static trampoline jumps to the prolog function. The prolog function uses the information provided by the static trampoline, sets things up for the ABI handler and then jumps to the ABI handler. - Call ffi_tramp_set_parms () in ffi_prep_closure_loc () to initialize static trampoline parameters. Signed-off-by: Madhavan T. Venkataraman <[email protected]>
|
Can you comment on the thread-safety of your patch? ffi_tramp_init in particular. We punt responsibility for thread-safety to the calling program, but we should at least add a note to the thread-safety section of the documentation. |
|
If I'm reading this patch right, it will try to use the static trampolines and fall back to the dynamic method if that fails. Why would static trampolines every not be available on a supported platform. I'm wondering if we can simply not include the old dynamic method. |
|
ffi_tramp_init() is a (static) internal function that is only called by ffi_tramp_is_supported() and ffi_tramp_alloc(). In both cases, the global static trampoline lock is acquired via ffi_tramp_lock(). In fact, all static trampoline operations acquire the lock. So, they are thread-safe. |
|
On the fallback question - the issue is in ffi_tramp_init_os (). This is the OS-specific initialization function. It needs to call OS-specific functions which can potentially fail. libffi must be resilient to that. E.g., let us consider loading the trampoline code in a temporary file and mapping it. Today, this is allowed. Tomorrow, an OS may require a valid file signature to allow mmap(file) with EXEC permissions. That will not work on temporary files. |
|
I have addressed all of the comments so far. Is there anything else I should address? If not, can these changes be merged into libffi? |
|
Thank you. I have merged this great improvement to libffi. The testing all looks good, so I hope there will be no problems. |
|
Thank you so much!
Madhavan
…On 3/5/21 10:09 AM, Anthony Green wrote:
Thank you. I have merged this great improvement to libffi. The testing all looks good, so I hope there will be no problems.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#624 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AR54PMXQDIYGGJP6Z3IRRMTTCD62DANCNFSM4XGXWZWQ>.
|
| uintptr_t addr = (uintptr_t) tramp_globals.text; | ||
| int nfields, found; | ||
|
|
||
| snprintf (file, PATH_MAX, "/proc/%d/maps", getpid()); |
There was a problem hiding this comment.
Hi. I'm curious if there is some case where /proc/self/maps does not work here. If there is, a comment would have been useful.
There was a problem hiding this comment.
maps has been around for a while and has remained stable AFAICT. However, I have assumed that it can fail. If it does, the code falls back to using a temporary file to map the trampoline table.
There was a problem hiding this comment.
I was curious enough to actually trace /proc/pid/maps to first appear in Linux 0.99.13k, but at that time it did not have the file name column. The file name column was at least in 2.4.12 if not even earlier. But I was thinking about use of snprintf and getpid being useless because can't you just open /proc/self/maps. Unless I miss something. /proc/self symlinks to getpid() so it should work the same. As it works the same, this is nothing major, just avoids any chance of mistakes with snprintf to stack. I checked /proc/self too, and it appeared in Linux-0.99 back in 1992.
There was a problem hiding this comment.
I agree. I could/should have used /proc/self/maps. That would have been more elegant.
There was a problem hiding this comment.
Ok. Thanks. My major concern was if there is something I did not know and you could tell me why it indeed makes sense to use getpid and snprintf.
To add, it is not just about signatures. Today, some security conscious have all their executable files on read only filesystems. If any mount is writable, it can be noexec mounted and there is no executable mmap from it. Some mount only /tmp noexec. |
|
Neat, thank you for the effort. |
|
Hi Anthony,
How are you?
Can you tell me what version of libffi this landed in? Our Yocto needs to be updated to the right version of libffi.
Thanks in advance for the information.
Madhavan
…On 3/5/21 10:07, Anthony Green wrote:
Merged #624 <#624> into master.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#624 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AR54PMXWZNXABYVQDV5QM73TCD6VHANCNFSM4XGXWZWQ>.
|
|
Hi Madhavan,
This was included in version 3.4.2.
AG
On Wed, Mar 2, 2022 at 2:04 PM Madhavan T. Venkataraman <
***@***.***> wrote:
… Hi Anthony,
How are you?
Can you tell me what version of libffi this landed in? Our Yocto needs to
be updated to the right version of libffi.
Thanks in advance for the information.
Madhavan
On 3/5/21 10:07, Anthony Green wrote:
> Merged #624 <#624> into master.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub <
#624 (comment)>, or
unsubscribe <
https://github.com/notifications/unsubscribe-auth/AR54PMXWZNXABYVQDV5QM73TCD6VHANCNFSM4XGXWZWQ
>.
>
—
Reply to this email directly, view it on GitHub
<#624 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAV7CK7SHDA5APZRV47K7TU563URANCNFSM4XGXWZWQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you modified the open/close state.Message
ID: ***@***.***>
|
For the benefit and technical details of static trampoline, see libffi#624. As a new architecture, let's be "safer" from the start. The change survived libffi testsuite on loongarch64-linux-gnu.
For the benefit and technical details of static trampoline, see #624. As a new architecture, let's be "safer" from the start. The change survived libffi testsuite on loongarch64-linux-gnu.
* Static Trampolines
Closure Trampoline Security Issue
=================================
Currently, the trampoline code used in libffi is not statically defined in
a source file (except for MACH). The trampoline is either pre-defined
machine code in a data buffer. Or, it is generated at runtime. In order to
execute a trampoline, it needs to be placed in a page with executable
permissions.
Executable data pages are attack surfaces for attackers who may try to
inject their own code into the page and contrive to have it executed. The
security settings in a system may prevent various tricks used in user land
to write code into a page and to have it executed somehow. On such systems,
libffi trampolines would not be able to run.
Static Trampoline
=================
To solve this problem, the trampoline code needs to be defined statically
in a source file, compiled and placed in the text segment so it can be
mapped and executed naturally without any tricks. However, the trampoline
needs to be able to access the closure pointer at runtime.
PC-relative data referencing
============================
The solution implemented in this patch set uses PC-relative data references.
The trampoline is mapped in a code page. Adjacent to the code page, a data
page is mapped that contains the parameters of the trampoline:
- the closure pointer
- pointer to the ABI handler to jump to
The trampoline code uses an offset relative to its current PC to access its
data.
Some architectures support PC-relative data references in the ISA itself.
E.g., X64 supports RIP-relative references. For others, the PC has to
somehow be loaded into a general purpose register to do PC-relative data
referencing. To do this, we need to define a get_pc() kind of function and
call it to load the PC in a desired register.
There are two cases:
1. The call instruction pushes the return address on the stack.
In this case, get_pc() will extract the return address from the stack
and load it in the desired register and return.
2. The call instruction stores the return address in a designated register.
In this case, get_pc() will copy the return address to the desired
register and return.
Either way, the PC next to the call instruction is obtained.
Scratch register
================
In order to do its job, the trampoline code would need to use a scratch
register. Depending on the ABI, there may not be a register available for
scratch. This problem needs to be solved so that all ABIs will work.
The trampoline will save two values on the stack:
- the closure pointer
- the original value of the scratch register
This is what the stack will look like:
sp before trampoline ------> --------------------
| closure pointer |
--------------------
| scratch register |
sp after trampoline -------> --------------------
The ABI handler can do the following as needed by the ABI:
- the closure pointer can be loaded in a desired register
- the scratch register can be restored to its original value
- the stack pointer can be restored to its original value
(the value when the trampoline was invoked)
To do this, I have defined prolog code for each ABI handler. The legacy
trampoline jumps to the ABI handler directly. But the static trampoline
defined in this patch jumps tp the prolog code which performs the above
actions before jumping to the ABI handler.
Trampoline Table
================
In order to reduce the trampoline memory footprint, the trampoline code
would be defined as a code array in the text segment. This array would be
mapped into the address space of the caller. The mapping would, therefore,
contain a trampoline table.
Adjacent to the trampoline table mapping, there will be a data mapping that
contains a parameter table, one parameter block for each trampoline. The
parameter block will contain:
- a pointer to the closure
- a pointer to the ABI handler
The static trampoline code would finally look like this:
- Make space on the stack for the closure and the scratch register
by moving the stack pointer down
- Store the original value of the scratch register on the stack
- Using PC-relative reference, get the closure pointer
- Store the closure pointer on the stack
- Using PC-relative reference, get the ABI handler pointer
- Jump to the ABI handler
Mapping size
============
The size of the code mapping that contains the trampoline table needs to be
determined on a per architecture basis. If a particular architecture
supports multiple base page sizes, then the largest supported base page size
needs to be chosen. E.g., we choose 16K for ARM64.
Trampoline allocation and free
==============================
Static trampolines are allocated in ffi_closure_alloc() and freed in
ffi_closure_free().
Normally, applications use these functions. But there are some cases out
there where the user of libffi allocates and manages its own closure
memory. In such cases, static trampolines cannot be used. These will
fall back to using legacy trampolines. The user has to make sure that
the memory is executable.
ffi_closure structure
=====================
I did not want to make any changes to the size of the closure structure for
this feature to guarantee compatibility. But the opaque static trampoline
handle needs to be stored in the closure. I have defined it as follows:
- char tramp[FFI_TRAMPOLINE_SIZE];
+ union {
+ char tramp[FFI_TRAMPOLINE_SIZE];
+ void *ftramp;
+ };
If static trampolines are used, then tramp[] is not needed to store a
dynamic trampoline. That space can be reused to store the handle. Hence,
the union.
Architecture Support
====================
Support has been added for aarch64. Support for other architectures can be
added very easily in the future.
OS Support
==========
Support has been added for Linux. Support for other OSes can be added very
easily.
* arm64: Support for Static Trampolines
- Define the arch-specific initialization function ffi_tramp_arch ()
that returns trampoline size information to common code.
- Define the trampoline code mapping and data mapping sizes.
- Define the trampoline code table statically.
- Introduce a tiny prolog for each ABI handling function. The ABI
handlers addressed are:
- ffi_closure_SYSV
- ffi_closure_SYSV_V
The prolog functions are called:
- ffi_closure_SYSV_alt
- ffi_closure_SYSV_V_alt
The legacy trampoline jumps to the ABI handler. The static
trampoline jumps to the prolog function. The prolog function uses
the information provided by the static trampoline, sets things up
for the ABI handler and then jumps to the ABI handler.
- Call ffi_tramp_set_parms () in ffi_prep_closure_loc () to
initialize static trampoline parameters.
Signed-off-by: Madhavan T. Venkataraman <[email protected]>
For the benefit and technical details of static trampoline, see libffi/libffi#624. As a new architecture, let's be "safer" from the start. The change survived libffi testsuite on loongarch64-linux-gnu.
Fixed a couple of build problems. Also excluded cygwin from static trampolines. This is ready for review as soon as the CI tests pass.