What is needed in uv's Python to allow static libpython build?

I'm one of the maintainers of the Python build used by uv (https://github.com/astral-sh/python-build-standalone). In f5e7640 you write

Static link library of UV-Python is currently using dependent libraries
such as tcl that are not included, but would be needed. Please help them
improve it for best performance of the result.

What is specifically needed here? The distribution used by uv, the "install-only" builds from python-build-standalone, doesn't ship a static libpython.a at all, and the dynamic libpython.so statically links all of its own dependencies including Tcl.

Can you expand on what is not working and what we would need to change? Thanks!

I am pretty sure I saw a libpython.a and Nuitka would love to use it, because DLL call overhead makes compiled programs only slower and prevents LTO inlining from happening, of course, only if said library includes gcc or clang bytecode. In Anaconda, if I recall correctly, that's much better for performance. But also for IP protection, a separate Python DLL makes a deployed binary very open for manipulation.

I was testing with 3.7 (a version definitely on my system at the time) and 3.13, and it seems you are correct about 3.13 about not having it, and probably everything not too old:

~/.local/share/uv/python$ find . -name libpython*
./cpython-3.13.1-linux-x86_64-gnu/lib/libpython3.13.so.1.0
./cpython-3.13.1-linux-x86_64-gnu/lib/libpython3.so
./cpython-3.13.1-linux-x86_64-gnu/lib/libpython3.13.so
./cpython-3.7.9-linux-x86_64-gnu/lib/libpython3.7m.so.1.0
./cpython-3.7.9-linux-x86_64-gnu/lib/libpython3.7m.a
./cpython-3.7.9-linux-x86_64-gnu/lib/libpython3.so
./cpython-3.7.9-linux-x86_64-gnu/lib/python3.7/config-3.7m-x86_64-linux-gnu/libpython3.7m.a
./cpython-3.7.9-linux-x86_64-gnu/lib/libpython3.7m.so
./cpython-3.13.0-linux-x86_64-gnu/lib/libpython3.13.so.1.0
./cpython-3.13.0-linux-x86_64-gnu/lib/libpython3.so
./cpython-3.13.0-linux-x86_64-gnu/lib/libpython3.13.so

This libpython.a would ideally be present and include everything the libpython.3.7.so statically includes of course.

Maybe the 3.7 build is with an older recipe, which I was not aware of is not typical of uv-Python. I made uv-python work with that version first, and I had to prevent the use of libpython3.7m.a actively, as it's otherwise suggested to be used.

I can see how you would hate to provide such a library as it adds overhead that is not commonly used, increasing installation time and download size. Maybe some sort of add-on package to download would be good there already.

Is there any hope for this? I really would appreciate it if there was proper support for static libpython in uv-Python, so I could recommend it over pyenv and Homebrew, that are moving targets and pretty hard to support.

busy trying to package an exe with nuitka and uv would love this to work

@WilliamStam

@WilliamStam https://discord.gg/NtUz4Xc9 join the server if you want to discuss, but it should work already.

Update to uv build system, add nuitka workaround

I think this has failed and won't happen. If I understand it correctly, the static libpython if it was provided would only work the a specific version of Clang that is not available for use from uv itself. So unlike Anaconda and such, there is no option to use it with gcc or other clang versions, let alone other compilers the user has on their system.

Making the C compiler part of the ABI is not acceptable to UV for all I know, so there is just no way. This also means that with using Nuitka for size/performance we have to recommend other flavors or self compiled Python with proper options.

If anybody can change that on standalone build python side, be my guest, but I can totally understand how that's undesirable.

I think this has failed and won't happen.
If I understand it correctly, the static libpython if it was provided would only work with a specific version of Clang that is not available for use from uv itself.

I believe you're mistaken? There is no clear example in these discussions for when that is the case?

Nvidia for example distributes static libraries (.a) in addition to their .so dynamic library equivalents. There is a rust crate cudarc that assist with linking these at build time for projects, as Nvidia does not provide sources to compile your own .o object files to link into a library/binary. So it shouldn't be all that different compatibility wise, like it is for a .so with min glibc version based on symbols used.

One example of what you might have had in mind is this more advanced LTO support in Rust which requires expects LLVM compatible bitcode between rust and clang to perform cross-language LTO.

In Fedora all you need to do is dnf install libatomic-static. A compiler may bundle their own libatomic and implicitly link that when appropriate, but AFAIK it's not a situation that'd be improved by preferring libatomic.so 🤔

UPDATE: On Fedora GCC provides a linker script .so stub, Clang also uses the one at the GCC location if present, the linker script points to /usr/lib64 from the libatomic package for a versioned shared lib.

Feedback - Collapsed for brevity (click to view)

Making the C compiler part of the ABI is not acceptable to UV for all I know, so there is just no way.

Where is this the case?

`python-build-standalone` prioritizes runtime compatibility over performance and size

This also means that with using Nuitka for size/performance we have to recommend other flavors or self compiled Python with proper options.

python-build-standalone is designed for broader compatibility. The current min glibc required is 2.17, as opposed to what you may get with a modern distro package of Python, this approach is convenient for developers to support users on older distros.

A side-effect of having glibc that low is it misses out on performance opportunities, such as close_range() when glibc 2.34 or newer is available (and a 5.9 or newer kernel). python-build-standalone is considering raising the min glibc version, but it's unlikely to go that high AFAIK, but a custom build could be done when that matters (presumably still compatible with Nuitka).

When performance matters or size matters, compiling Python from source would obviously have an advantage, but again that should be dependent upon your demographic. In OCI/Docker images for example, various CPU instructions/features are assumed not available by default, but you could build and publish for x86_64-v3 (or v4) instead of the rather old v1.

`libatomic.a` shouldn't be an issue

If anybody can change that on standalone build python side, be my guest, but I can totally understand how that's undesirable.

It looks like more of an issue on Nuitka's side IMO, it would be good to know why you think libatomic.a should be used? I've tried to get an overall picture of what Nuitka's involvement is, but I'm really not sure why it's even doing what it does in the first place to cause the libatomic.a issue 🤷‍♂️

Personally, unless someone can actually clarify differently, a static dep isn't that big of an issue when building software (I tend to prefer it at times). -l atomic would also be valid, in that IIRC a static .a lib will be used if it's found in the search path before a .so, there's often a configurable to enforce dynamic/static linking too.

`libpython.a` should only be present via opt-in

This libpython.a would ideally be present and include everything the libpython.3.7.so statically includes of course.

That would be a bit annoying/redundant for anyone who doesn't need the extra weight.

.a / static libs are build-time deps, just like Fedora and other distros provide via -static or similar package variants. When only needing a static lib to build, having the dynamic lib .so would be redundant, so distribution for a static lib should be handled separately via opt-in IMO.

One perk with .so is the user/dev can easily tell what deps are required (unless they're implicit at runtime via dlopen() call). I don't recall that being obvious with .a, since that's link-time and only cares about symbols being resolved?

Is there any hope for this? I really would appreciate it if there was proper support for static libpython in uv-Python

I've only had a brief glance over at python-build-standalone, but one would assume with all the .o artifacts built, they could archive those into a .a easily enough 🤷‍♂️

NOTE: There is this comment about stripping an -fPIC flag from a build.

I recall an issue when that is omitted, at least if you were to build a proper static executable (with a static relocation model rather than PIE aka static-pie) which expects any .a to also have been built with -fPIC (EDIT: Actually, I might be recalling wrong and it was to do with building a .so with -shared where -fPIC is required, and thus any objects linked to it?), and I recall that being a bit tricky to infer in advance if you only had the .a at hand.

Unlikely to make a difference with manipulation concern?

But also for IP protection, a separate Python DLL makes a deployed binary very open for manipulation.

If your binary is relying on dlopen(), you're going to be using glibc for example, anything linked (or library loaded at runtime via dlopen()) is then just as prone to manipulation via LD_PRELOAD or similar no?

Building a single executable relying on an interpreter (eg: /lib64/ld-linux-x86-64.so.2), that can be hooked with it's dlopen() call to glibc to initialize the program.

Beyond that, what other attacks are you differentiating a static libpython lib would prevent?

Original Message (click to view)

If anybody can change that on standalone build python side, be my guest, but I can totally understand how that's undesirable.

libatomic.a is a build-time requirement for python-build-standalone, specifically for an extension with an OpenSSL 3.x dependency.

libatomic.a as a link-time dep instead of run-time dep (libatomic.so) was intended to simplify deployment to the end-user as no additional dependency would need to be provided by the runtime system:

This issue notes that an unfortunate side-effect was the libatomic.a dep was leaking through the build process unintentionally:

unfortunately the Python extension build process leaks flags from how Python itself was built, which is probably why you're getting that on your build command line.

UPDATE: I've since detailed this leakage below, and how Nuitka is incorrectly attempting to link libatomic.a.

UPDATE2: This comment details the upstream concern for why it's -l:libatomic.a instead of -l atomic.

I think this has failed and won't happen. If I understand it correctly, the static libpython if it was provided would only work the a specific version of Clang that is not available for use from uv itself.

Reference: Newer comment by @kayhayen at related python-build-standalone discussion

This would be similar to glibc / libc.so.6 and GCC's libgcc_s.so.1 as common deps that may be linked if you were to do static linking with their .a equivalents.

Glibc is one of the fewer deps where static linking is unwise / problematic. However building with libatomic.a should not be too different to linking libatomic.so between build host and runtime environment?

You should avoid linking multiple different copies though, similar to how it would be bad to statically link your own glibc into a shared library whilst your main program uses the system glibc. It would seem odd that you require libatomic.a when there is no libpython.a involved 🤷‍♂️ (you can technically unpack these to bundle the .o files they contain IIRC, a .a file is effectively just an archive of .o files AFAIK)

So this appears to be a misunderstanding on Nuitka's end?

UPDATE: Confirmed below. Nuitka is discovering link information and using that for some reason redundantly? (I'm not familiar enough with the project but it seems like an odd behaviour)

`libatomic.a` is not needed

For example using Nuitka with python-build-standalone supplied from UV we can reproduce the build failure (when libatomic.a is not available) and see the command invoked:

# Basic project via `uv init --python 3.10`:
$ uv run nuitka --standalone main.py
$ head -n 5 nuitka-crash-report.xml

<?xml version='1.0' encoding='utf8'?>
<nuitka-compilation-report nuitka_version="2.8rc17" nuitka_commercial_version="not installed" mode="standalone" completion="error exit message (1)" exit_message="Failed unexpectedly in Scons C backend compilation.">
  <scons_error_reports>
    <scons_error_report>
      <command>gcc -o "/example/main.dist/main.bin" -s -z noexecstack -export-dynamic -Wl,-rpath=/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib -Wl,-R,'$ORIGIN' -Wl,--disable-new-dtags @"./@link_input.txt" -L/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib -ldl -lm -lpython3.10 -ldl -lpthread -lutil -lrt -lm -l:libatomic.a -l:libatomic.a</command>

Remove the two -l:libatomic.a -l:libatomic.a at the end of that command (which Nuitka has presumably appended somewhere?) and then run that command from within the /example/main.build/ dir and this basic main.py hello world example builds successfully:

$ cd main.build/

$ gcc -o "/example/main.dist/main.bin" -s -z noexecstack -export-dynamic -Wl,-rpath=/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib -Wl,-R,'$ORIGIN' -Wl,--disable-new-dtags @"./@link_input.txt" -L/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib -ldl -lm -lpython3.10 -ldl -lpthread -lutil -lrt -lm

$ ../main.dist/main.bin
Hello from example!

libatomic.a wasn't actually required at all 🤔

Nuitka has wrongly attempted to link it (twice for some reason? Same with libdl + libm), but perhaps Nuitka just doesn't know any better and is trusting wherever that information for linking is sourced from?

Same story with Clang:

# CFLAGS `-w` is set to omit outputting warnings with Clang,
# this is more consistent with the crash report written since the warnings aren't collected as "errors"
CFLAGS='-w' uv run nuitka --standalone --clang main.py

$ head -n 5 nuitka-crash-report.xml
<?xml version='1.0' encoding='utf8'?>
<nuitka-compilation-report nuitka_version="2.8rc17" nuitka_commercial_version="not installed" mode="standalone" completion="error exit message (1)" exit_message="Failed unexpectedly in Scons C backend compilation.">
  <scons_error_reports>
    <scons_error_report>
      <command>clang -o "/example/main.dist/main.bin" -z noexecstack -Wl,--export-dynamic -Wl,-rpath=/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib -Wl,-R,'$ORIGIN' -Wl,--disable-new-dtags @"./@link_input.txt" -L/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib -ldl -lm -lpython3.10 -ldl -lpthread -lutil -lrt -lm -l:libatomic.a -l:libatomic.a</command>

Rpath to original Python path (Resolved)

As is evident from the build command, the rpath configured is not ideal:

$ patchelf --print-rpath /example/main.dist/main.bin
/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib:$ORIGIN

But this appears to be corrected to $ORIGIN at a later stage (if libatomic.a was available for a successful build):

First rpath addition via here:

Nuitka/nuitka/build/Backend.scons

Lines 609 to 614 in a43a4a0

    
           # For NetBSD the rpath is required, on FreeBSD it's warned as unused, macOS 
        
           # and AIX don't have it, but some Python flavors like Standalone Python 
        
           # won't find it unless provided like this, so lets try to add this by 
        
           # default and remove OSes that give errors one by one. 
        
           if not isFreeBSD() and not isMacOS() and not isAIX(): 
        
               env.Append(LINKFLAGS=["-Wl,-rpath=" + python_lib_path])

Second rpath added here:

Nuitka/nuitka/build/Backend.scons

Lines 654 to 677 in a43a4a0

    
           # Set load libpython from binary directory default 
        
           if ( 
        
               env.gcc_mode 
        
               and not isMacOS() 
        
               and not isAIX() 
        
               and not os.name == "nt" 
        
               and (env.exe_mode or env.dll_mode) 
        
           ): 
        
               # TODO: It would appear that this should be merged with above code that does 
        
               # env.Append(LINKFLAGS=["-Wl,-rpath=" + python_lib_path]) relatively 
        
               # unconditionally already. 
        
               if env.standalone_mode: 
        
                   rpath = "$$ORIGIN" 
        
               else: 
        
                   rpath = python_lib_path 
        
               env.Append(LINKFLAGS=["-Wl,-R,'%s'" % rpath]) 
        
               # The rpath is no longer used unless we do this on modern Linux. The 
        
               # option name is not very revealing, but basically without this, the 
        
               # rpath in the binary will be ignored by the loader. 
        
               # spell-checker: ignore dtags 
        
               if isLinux(): 
        
                   env.Append(LINKFLAGS=["-Wl,--disable-new-dtags"])

NOTE: Clang or GCC detection applies the rpath shown above:

Nuitka/nuitka/build/Backend.scons

Lines 349 to 362 in a43a4a0

    
           # Requested or user provided, detect if it's clang even from environment 
        
           if isClangName(env.the_cc_name): 
        
               clang_mode = True 
        
               env["CCVERSION"] = None 
        
           # We consider clang to be a form of gcc for the most things, they strive to 
        
           # be compatible. 
        
           env.gcc_mode = isGccName(env.the_cc_name) or clang_mode 
        
           env.clang_mode = clang_mode 
        
           # Only use MSVC if not already clear, we are using MinGW. 
        
           env.msvc_mode = os.name == "nt" and not env.gcc_mode 
        
           env.mingw_mode = os.name == "nt" and env.gcc_mode 
        
           env.clangcl_mode = clangcl_mode

Rpath is corrected after successfully linking dist binary:

Nuitka/nuitka/freezer/Standalone.py

Lines 202 to 206 in a43a4a0

    
           # After dependency detection, we can change the RPATH for main binary. 
        
           if isRPathUsingPlatform(): 
        
               setSharedLibraryRPATH( 
        
                   os.path.join(dist_dir, standalone_entry_points[0].dest_path), "$ORIGIN" 
        
               )

Investigation

EDIT: Since neither discussion pointed out these details (perhaps it was obvious to both Nuitka and python-build-standalone devs?) Here is why Nuitka is linking libatomic.a:

$ grep -r libatomic /root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu

grep: /root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib/python3.10/__pycache__/_sysconfigdata__linux_x86_64-linux-gnu.cpython-310.pyc: binary file matches
/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py:    "LOCALMODLIBS": "-lbz2 -lffi -ldl -lncursesw -lpanelw -lncursesw -ldb -lmpdec -lexpat -lcrypto -l:libatomic.a -llzma -lrt -lsqlite3 -lssl -lcrypto -l:libatomic.a -luuid -lm -lm -lexpat -ledit -lncursesw -lz",
/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py:    "MODLIBS": "-lbz2 -lffi -ldl -lncursesw -lpanelw -lncursesw -ldb -lmpdec -lexpat -lcrypto -l:libatomic.a -llzma -lrt -lsqlite3 -lssl -lcrypto -l:libatomic.a -luuid -lm -lm -lexpat -ledit -lncursesw -lz",
/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile:LOCALMODLIBS=     -lbz2          -lffi -ldl  -lncursesw  -lpanelw -lncursesw   -ldb  -lmpdec  -lexpat  -lcrypto -l:libatomic.a     -llzma       -lrt          -lsqlite3  -lssl -lcrypto -l:libatomic.a        -luuid        -lm    -lm    -lexpat  -ledit -lncursesw         -lz
/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile:Modules/_hashlib$(EXT_SUFFIX):  Modules/_hashopenssl.o; $(BLDSHARED)  Modules/_hashopenssl.o  -lcrypto -l:libatomic.a  -o Modules/_hashlib$(EXT_SUFFIX)
/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile:Modules/_ssl$(EXT_SUFFIX):  Modules/_ssl.o; $(BLDSHARED)  Modules/_ssl.o  -lssl -lcrypto -l:libatomic.a  -o Modules/_ssl$(EXT_SUFFIX)
/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib/python3.10/config-3.10-x86_64-linux-gnu/Setup.local:_hashlib _hashopenssl.c -I/tools/deps/include -lcrypto -l:libatomic.a
/root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib/python3.10/config-3.10-x86_64-linux-gnu/Setup.local:_ssl _ssl.c -I/tools/deps/include -lssl -lcrypto -l:libatomic.a

More readable summary:

lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py has dictionary keys LOCALMODLIBS + MODLIBS from the build_time_vars var.
lib/python3.10/config-3.10-x86_64-linux-gnu/{Makefile,Setup.local} files have matches above for the two mentioned extensions Modules/_hashopenssl.o + Modules/_ssl.o, along with LOCALMODLIBS assignment in the Makefile.

LOCALMODLIBS and MODLIBS have the same value (and this is where the double libatomic.a is coming from):

-lbz2 -lffi -ldl -lncursesw -lpanelw -lncursesw -ldb -lmpdec -lexpat -lcrypto -l:libatomic.a -llzma -lrt -lsqlite3 -lssl -lcrypto -l:libatomic.a -luuid -lm -lm -lexpat -ledit -lncursesw -lz

MODLIBS is sourced by Nuitka here:

Nuitka/nuitka/PythonVersions.py

Lines 493 to 504 in a43a4a0

    
           def getModuleLinkerLibs(): 
        
               """Get static link libraries needed.""" 
        
               try: 
        
                   import sysconfig 
        
               except ImportError: 
        
                   return [] 
        
               else: 
        
                   # static link libraries might be there, spell-checker: ignore modlibs 
        
                   result = sysconfig.get_config_var("MODLIBS") or "" 
        
                   result = [entry[2:] for entry in result.split() if entry.startswith("-l:")] 
        
                   return result

Nuitka/nuitka/MainControl.py

Lines 739 to 741 in a43a4a0

    
           link_module_libs = getModuleLinkerLibs() 
        
           if link_module_libs: 
        
               scons_options["link_module_libs"] = ",".join(link_module_libs)

Nuitka/nuitka/build/Backend.scons

Lines 547 to 548 in a43a4a0

    
           # Any module libs that are for self-compiled Python to be static. 
        
           env.Append(_LIBFLAGS=["-l" + lib_desc for lib_desc in link_module_libs])

Why is this linking logic triggered?

Given everything I've shared above, I have to wonder why static libs are being collected for a dynamic library of libpython?

Nuitka is linking with -lpython3.10, so if that library were correctly built, why is all of this necessary for the built binary? (EDIT: actually based on the earlier comment about Python 3.7 .a, it seems to be trying to support the possibility of .a being found before an equivalent .so, if any_)

Inspecting the linkage:

# Full resolution (NOTE: rpath was incorrectly configured):
$ ldd ../main.dist/main.bin
        linux-vdso.so.1 (0x00007fff7a594000)
        libm.so.6 => /lib64/libm.so.6 (0x00007d170b262000)
        libpython3.10.so.1.0 => /root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib/libpython3.10.so.1.0 (0x00007d170a1c5000)
        libc.so.6 => /lib64/libc.so.6 (0x00007d1709fd3000)
        /lib64/ld-linux-x86-64.so.2 (0x00007d170b355000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007d1709fcf000)
        librt.so.1 => /lib64/librt.so.1 (0x00007d1709fc9000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007d1709fc5000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00007d1709fc1000)

# Actual direct deps linked:
$ patchelf --print-needed ../main.dist/main.bin
libm.so.6
libpython3.10.so.1.0
libc.so.6

# UV python:
$ patchelf --print-needed /root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib/libpython3.10.so.1.0
libdl.so.2
librt.so.1
libm.so.6
libpthread.so.0
libutil.so.1
libc.so.6

$ du --bytes /root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib/libpython3.10.so.1.0
19002600        /root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/lib/libpython3.10.so.1.0

# Nuitka's modified copy added to `main.build/` (if linking of `main.bin` was successful):
$ patchelf --print-needed main.dist/libpython3.10.so.1.0
libdl.so.2
librt.so.1
libm.so.6
libpthread.so.0
libutil.so.1
libc.so.6

# Almost 60KB larger:
$ du --bytes main.dist/libpython3.10.so.1.0
19061697        main.dist/libpython3.10.so.1.0

Besides setting the rpath to $ORIGIN (which apparently python-build-standalone does as well, but for whatever reason is unset for the copy UV installed) and not having executable permissions (which ldd warns about), I've not looked through Nuitka source enough to see if anything else differs with the copy of libpython into the dist dir?

Proper approach

The following sections will use libatomic as an example reference for understanding this properly, but for the most-part, you should be able to do the following:

$ ld -shared -L path/to/dir/with/libpython -l:libpython.a -o /dev/null
ld: cannot find -l:libpython.a: No such file or directory

# Failed exit status returned:
$ echo $?
1

If that actually succeeded, then you'd append the link deps from MODLIBS.

`libatomic.so` vs `libatomic.a` availability in packages

Specifically with Fedora as a reference, depending on distro and static lib needed, not all provide a package which can be a point of friction if not using a container to build.

$ docker run --rm -it fedora:42 bash

# Show packages with `libatomic.a`:
$ dnf provides '*/libatomic.a'

gcc-15.2.1-1.fc42.x86_64 : Various compilers (C, C++, Objective-C, ...)
Repo         : @System
Matched From :
Filename     : /usr/lib/gcc/x86_64-redhat-linux/15/32/libatomic.a

libatomic-static-15.2.1-1.fc42.x86_64 : The GNU Atomic static library
Repo         : @System
Matched From :
Filename     : /usr/lib/gcc/x86_64-redhat-linux/15/libatomic.a

gcc-offload-nvptx-15.2.1-1.fc42.x86_64 : Offloading compiler to NVPTX
Repo         : updates
Matched From :
Filename     : /usr/lib/gcc/x86_64-redhat-linux/15/accel/nvptx-none/libatomic.a
Filename     : /usr/lib/gcc/x86_64-redhat-linux/15/accel/nvptx-none/mgomp/libatomic.a

gcc14-14.2.1-8.fc42.x86_64 : Various compatibility compilers (C, C++, Fortran, ...)
Repo         : fedora
Matched From :
Filename     : /usr/lib/gcc/x86_64-redhat-linux/14/libatomic.a

# ...

Here libatomic.so is implicitly available with GCC, which is handy as a developer but for runtime is an extra dep the user should have present (unless you bundle it):

# No shared deps, perfect for static linking with the `libatomic.a` instead:
$ ldd /usr/lib/gcc/x86_64-redhat-linux/15/libatomic.so
ldd: warning: you do not have execution permission for `/usr/lib/gcc/x86_64-redhat-linux/15/libatomic.so'
        not a dynamic executable


# `libatomic.so` can be found by a few packages:
# GCC for the direct `.so` stub, while versioned lib is primarily from `libatomic` package
$ dnf provides '*/libatomic.so*'

gcc-15.2.1-1.fc42.x86_64 : Various compilers (C, C++, Objective-C, ...)
Repo         : @System
Matched From :
Filename     : /usr/lib/gcc/x86_64-redhat-linux/15/32/libatomic.so
Filename     : /usr/lib/gcc/x86_64-redhat-linux/15/libatomic.so

libatomic-15.2.1-1.fc42.x86_64 : The GNU Atomic library
Repo         : @System
Matched From :
Filename     : /usr/lib64/libatomic.so.1
Filename     : /usr/lib64/libatomic.so.1.2.0

gcc14-14.2.1-8.fc42.x86_64 : Various compatibility compilers (C, C++, Fortran, ...)
Repo         : fedora
Matched From :
Filename     : /usr/lib/gcc/x86_64-redhat-linux/14/libatomic.so

julia-1.11.0-13.rc3.fc42.x86_64 : High-level, high-performance dynamic language for technical computing
Repo         : fedora
Matched From :
Filename     : /usr/lib64/julia/libatomic.so.1

# ...

Inspecting the lib discovery process (link-time vs run-time)

In the case of libatomic.so.1, this is the actual filename that would be linked (and shown via ldd / patchelf --print-needed binary-filename-here).

For those unaware, it's not uncommon to see misunderstandings with link-time phase of builds where the linker searches for libatomic.so (unversioned .so files are often from devel packages or stubs to support finding libs at link time), but at runtime expects a versioned .so (based on the shared libraries DT_SONAME header which ldconfig can use and generate symlink(s) for when there is a mismatch to the filename).

For the GCC libatomic.so provided file, it's not a symlink but it's not a proper library either. You can see it's only 40 bytes in size with the text content of INPUT with a path value (thus it's a linker script):

$ docker run --rm -it fedora:42 bash
$ dnf install -yq gcc libatomic

$ du --bytes /usr/lib/gcc/x86_64-redhat-linux/15/libatomic.so
40      /usr/lib/gcc/x86_64-redhat-linux/15/libatomic.so

# It's a linker script (pointing to where the actual library is expected to be):
$ cat /usr/lib/gcc/x86_64-redhat-linux/15/libatomic.so
INPUT ( /usr/lib64/libatomic.so.1.2.0 )

# The actual library is the `libatomic.so.1.2.0` file (with a `libatomic.so.1` symlink):
$ ls -l /usr/lib64/libatomic.so*
lrwxrwxrwx 1 root root    18 Aug  8 00:00 /usr/lib64/libatomic.so.1 -> libatomic.so.1.2.0
-rwxr-xr-x 1 root root 37528 Aug  8 00:00 /usr/lib64/libatomic.so.1.2.0

# `ld.so` resolution can be checked via `ldconfig` (only relevant for runtime lib discovery):
# NOTE: For most cases `/etc/ld.so.cache` (updated by dirs listed in `/etc/ld.so.conf` config) is equivalent to `LD_LIBRARY_PATH` functionality
$ ldconfig -N --verbose 2>/dev/null | grep atomic
        libatomic.so.1 -> libatomic.so.1.2.0

# Check the `DT_SONAME`, notice the value returned differs from the filename:
# (hence `ldconfig` would create this symlink if it were missing)
$ patchelf --print-soname /usr/lib64/libatomic.so.1.2.0
libatomic.so.1

Let's see how the libatomic.so library can be found and checked for existence:

$ docker run --rm -it fedora:42 bash
$ dnf install -yq gcc

# If there is a `libatomic.so` or `libatomic.a` in the search path this wouldn't fail like below:
$ ld -shared -l atomic -o /dev/null
ld: cannot find -latomic: No such file or directory

# We know GCC package bundled the `libatomic.so`, and it will add that to the search path when invoked:
# However as shown above, this is a stub that links to a versioned lib (that is presently missing):
$ gcc -shared -nostdlib -l atomic -o /dev/null
/usr/sbin/ld: cannot find /usr/lib64/libatomic.so.1.2.0: No such file or directory

# Add the runtime shared library:
$ dnf install -yq libatomic

# `ld` still cannot locate it as the `.so` still is not part of the standard search paths:
# NOTE:
# - Only `libatomic.so.1` + `libatomic.so.1.2.0` exist in the search path
# - Optionally this can be fixed by adding the additional search path: `-L /usr/lib/gcc/x86_64-redhat-linux/15`
$ ld -shared -l atomic -o /dev/null
ld: cannot find -latomic: No such file or directory

# But the compiler `gcc` can now resolve it's linker script stub successully:
$ gcc -shared -nostdlib -l atomic -o /dev/null
# No output, but successful exit status:
$ echo $?
0

# Optionally output an actual linked binary and you can identify the link was successful:
$ gcc -shared -nostdlib -l atomic -o /tmp/example

# Library has linked to major versioned name to discover at runtime:
$ patchelf --print-needed /tmp/example
libatomic.so.1

# Which is then queried and resolved (`ldconfig` / `LD_LIBRARY_PATH` is relevant here):
$ ldd /tmp/example
        linux-vdso.so.1 (0x00007ffddc95f000)
        libatomic.so.1 => /lib64/libatomic.so.1 (0x00007f3a49877000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f3a49685000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f3a4988a000)

Clang will also implicitly use that GCC location to find libatomic.so / libatomic.a 👍
ld or other compilers like Zig would need the -L hint to discover it.

Static or Dynamic?

Now with all that out of the way, the actual concern in Nuitka linking logic was if -l libpython was going to find a .a for static linking (requiring it's deps linked alongside it), or if it's an already built shared dynamic library (.so) which doesn't need it's linkage applied to the output binary executable.

Rather simple, only apply the additional -l flags related to libpython if we're linking statically. We can use the linker flags --Bstatic to enforce future -l to be statically linked, and if any dynamic deps would follow that, add --Bdynamic again before the -l for dynamic links are introduced. Or you can be explicit with the full filename to search for (-l:libatomic.a vs -l:libatomic.so), which is supported by gcc but not ld.

$ docker run --rm -it fedora:42 bash
# `libatomic-static` brings in `libatomic` as well, so both `.a` + `.so.1` are present,
# in addition to the `.so` from the `gcc` package.
$ dnf install -yq gcc libatomic-static

# Explicitly force static/dynamic by filename:
$ ld -shared -L /usr/lib/gcc/x86_64-redhat-linux/15 -l:libatomic.a -o /dev/null
$ ld -shared -L /usr/lib/gcc/x86_64-redhat-linux/15 -l:libatomic.so -o /dev/null

# Explicitly force static or dynamic lib discovery:
# (rather than the first found `.so` or `.a` file match, where `.so` is checked first)
$ ld -shared -L /usr/lib/gcc/x86_64-redhat-linux/15 --Bstatic -l atomic -o /dev/null
$ ld -shared -L /usr/lib/gcc/x86_64-redhat-linux/15 --Bdynamic -l atomic -o /dev/null

# Without forcing static/dynamic discover `.so` would be preferred first when querying each search path:
$ ld -shared -L /usr/lib/gcc/x86_64-redhat-linux/15 -l atomic -o /tmp/example
$ patchelf --print-needed /tmp/example
libatomic.so.1

# Or without the added search dir, if you know the library to match at a default search path like `/usr/lib64`:
# (both still use the `DT_SONAME` entry of `libatomic.so.1` for `ld.so` to query at runtime)
$ ld -shared -l:libatomic.so.1 -o /dev/null
$ ld -shared -l:libatomic.so.1.2.0 -o /dev/null

For that specific GCC path in this case, you could identify it from this output:

$ gcc -print-search-dirs

install: /usr/lib/gcc/x86_64-redhat-linux/15/
programs: =/usr/libexec/gcc/x86_64-redhat-linux/15/:/usr/libexec/gcc/x86_64-redhat-linux/15/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/15/:/usr/lib/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/15/../../../../x86_64-redhat-linux/bin/x86_64-redhat-linux/15/:/usr/lib/gcc/x86_64-redhat-linux/15/../../../../x86_64-redhat-linux/bin/
libraries: =/usr/lib/gcc/x86_64-redhat-linux/15/:/usr/lib/gcc/x86_64-redhat-linux/15/../../../../x86_64-redhat-linux/lib/x86_64-redhat-linux/15/:/usr/lib/gcc/x86_64-redhat-linux/15/../../../../x86_64-redhat-linux/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/15/../../../x86_64-redhat-linux/15/:/usr/lib/gcc/x86_64-redhat-linux/15/../../../../lib64/:/lib/x86_64-redhat-linux/15/:/lib/../lib64/:/usr/lib/x86_64-redhat-linux/15/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/15/../../../../x86_64-redhat-linux/lib/:/usr/lib/gcc/x86_64-redhat-linux/15/../../../:/lib/:/usr/lib/

So taking that install path:

$ gcc -print-search-dirs | grep ^install: | cut -d' ' -f2
/usr/lib/gcc/x86_64-redhat-linux/15/

# Dynamically get the GCC toolchain path:
$ ld -shared -L "$(gcc -print-search-dirs | grep ^install: | cut -d' ' -f2)" \
  --Bstatic -l atomic -o /dev/null

# Success:
$ echo $?
0

The libatomic was not the issue I had, but it is with the static link libraries available as part of the download. They have all parts available, but not usable.

I am not saying there is no configuration in which their self built libraries cannot be replaced with OS libraries that are compatible with the OS C compiler. I am saying the static link libraries provided are not usable to me.

If then Nuitka is ran on Arch, Fedora, Debian, or whatever, I cannot be the maintainer of attempting to produce replacement libraries. They will be changing their toolchain a lot I assume, making it a huge effort to replace that. If anybody, UV would be the one to do that to begin with, but I understand how just building it themselves solves it so nicely that they won't care.

But unlike with something like Anaconda, I cannot have their toolchain it as it's not part of the ABI in the case of UV-Python downloads.

Also what you then produce is not what you can test with Python, as that's going to use different versions of everything, since e.g. the libatomic might behave differently in whatever way. I don't think I want people that have their resulting program not working claim it's a Nuitka bug what these differences may do to their programs.

And from what I recall, what they offer for downloads are not even fat objects, but specific compiler bytecode only. That is defeating much of the purpose of static linking and disallows link time optimization unless you have a matching C compiler (which they don't provide) and that most often the system C compiler cannot use.

As a side note: For portable binaries you want to build on old systems btw. I am using a Linux container image tool for my commercial users that I mean to make public, but lack the time, but using newest Fedora is inherently going to make it harder to get stuff that works on other systems as well.

Apologies for the verbosity in advance, I wanted to be thorough.

I still don't grasp the problem here? Are you certain there actually is one?
Even if a libpython.a were to be provided through UV, as shown below I fail to validate any of your concerns expressed.

So to recap:

python-build-standalone with a libpython.a or libpython.so should have no support concerns AFAIK, regarding toolchain use or compatibility across distros. If you can clarify this better I would be happy to verify, but presently it seems to be speculation?
Old systems for build environments should not be necessary either. If Nuitka was to fix it's Zig support as a compiler it should do the task just as well.
Nuitka should not be linking libatomic.a, it should change this to -l atomic or ignore it like it does with the ambiguous links of other deps in sysconfig MODLIBS (-l my-library does not always mean dynamically link).

Full response

I am saying the static link libraries provided are not usable to me.

What static libraries provided are you thinking about?

$ docker run --rm -it fedora:42 bash
$ dnf install -yq fd-find uv

$ uv python install 3.13
# Recursively search for any file with `.a` extension:
$ fd --extension 'a' . /root/.local/share/uv/
/root/.local/share/uv/python/cpython-3.13.6-linux-x86_64-gnu/lib/itcl4.2.4/libitclstub4.2.4.a

There is no libpython.a if that's what you were referring to.

The libatomic.a dep is not necessary and Nuitka is at fault for enforcing that when it's already a part of the libpython.so. I've communicated to upstream to remove the explicit static link, where their CI can still link statically and as a bonus Nuitka will be implicitly "fixed".

Nuitka should not be linking libatomic.a in the first place. Only if Nuitka was aware of -l python resolving to libpython.a, which I've already demonstrated above can be done, would you then need to do that.

Nuitka is perhaps trying to be smart for the users benefit, but in this case it's doing the wrong thing.

Earlier you did mention Python 3.7 was provided a static library. I am unable to reproduce that however 😓

The latest 0.9 release of UV offers a stable release of CPython 3.14, so it seems that version availability is dependent upon the version of UV? As such 3.7 is no longer available for me to retrieve, 3.8 is the oldest option I can use:

$ uv --version
uv 0.8.11

$ uv python install 3.7
error: No download found for request: cpython-3.7-linux-x86_64-gnu

$ uv python list --only-downloads

cpython-3.14.0rc2-linux-x86_64-gnu                 <download available>
cpython-3.14.0rc2+freethreaded-linux-x86_64-gnu    <download available>
cpython-3.13.6-linux-x86_64-gnu                    <download available>
cpython-3.13.6+freethreaded-linux-x86_64-gnu       <download available>
cpython-3.12.11-linux-x86_64-gnu                   <download available>
cpython-3.11.13-linux-x86_64-gnu                   <download available>
cpython-3.10.18-linux-x86_64-gnu                   <download available>
cpython-3.9.23-linux-x86_64-gnu                    <download available>
cpython-3.8.20-linux-x86_64-gnu                    <download available>
pypy-3.11.13-linux-x86_64-gnu                      <download available>
pypy-3.10.16-linux-x86_64-gnu                      <download available>
pypy-3.9.19-linux-x86_64-gnu                       <download available>
pypy-3.8.16-linux-x86_64-gnu                       <download available>
graalpy-3.11.0-linux-x86_64-gnu                    <download available>
graalpy-3.10.0-linux-x86_64-gnu                    <download available>
graalpy-3.8.5-linux-x86_64-gnu                     <download available>

The earlier 3.8 release has a few other static libs present:

$ uv python install 3.8
$ fd --extension 'a' . /root/.local/share/uv/ | grep 3.8

/root/.local/share/uv/python/cpython-3.8.20-linux-x86_64-gnu/lib/Tix8.4.3/libTix8.4.3.a
/root/.local/share/uv/python/cpython-3.8.20-linux-x86_64-gnu/lib/itcl4.2.2/libitcl4.2.2.a
/root/.local/share/uv/python/cpython-3.8.20-linux-x86_64-gnu/lib/itcl4.2.2/libitclstub4.2.2.a
/root/.local/share/uv/python/cpython-3.8.20-linux-x86_64-gnu/lib/thread2.8.7/libthread2.8.7.a

None of these are needed for the .so however, and there is no .a available for libpython.

The libatomic was not the issue I had, but it is with the static link libraries available as part of the download. They have all parts available, but not usable.

If then Nuitka is ran on Arch, Fedora, Debian, or whatever, I cannot be the maintainer of attempting to produce replacement libraries. They will be changing their toolchain a lot I assume, making it a huge effort to replace that.

Sorry, I'm a bit lost here and would like to understand your perspective better.

UV / python-build-standalone provided libpython is an already compiled binary .so dynamically library. I understand that you would prefer a libpython.a but that doesn't really change much 🤔

libpython.a: All static libs included:
- Except libatomic.a (presumably not built from source like the other bundled deps, so is required separately at link-time regardless)
- You must also dynamically link any other deps, such as glibc. No different to distributing a prebuilt libpython.so.
- Regardless of dynamic vs static linking of these additional libraries, you must have the symbols to link. ABI compatibility is relevant.
libpython.so:
- Already built as a separate library to dynamically link. Just requires those linked shared libraries to be present on the system.
- The shared libs are all from glibc in this case, so the only compatibility concern is with the symbols used. As glibc 2.17 was used at compilation, this isn't really a concern.

I do understand the additional benefits of a static library, and what you can do with that. But I am not seeing any compatibility issues?

I know software like Rspamd has dependencies linked that doesn't make it particularly portable IIRC, but python-build-standalone does not present with those problems.

I see no toolchain dependency that will cause breakage, if you have a more concrete example of where this is the case I'd be happy to know about that. It's rather easy to reproduce with containers once you've supplied that information, otherwise it seems like you've misunderstood.

Regarding libatomic the DT_SONAME linked is libatomic.so.1, no different really than OpenSSL 1.x vs 3.x, should a breaking change be introduced, you'd get that major version of the library bumped. Only then would that be a relevant concern, and only if you were to be concerned about linking it yourself as opposed to what has been done with statically linking it into the libpython.so built by python-build-standalone.

All other libraries statically linked are a non-concern compatibility wise, we already know what is required via the remaining shared libraries linked. One could statically link glibc, but that is unwise and not truly static.

Also what you then produce is not what you can test with Python, as that's going to use different versions of everything, since e.g. the libatomic might behave differently in whatever way.

Here is Fedora 30 (April 2019), although I'm sure you could go further back and still have success:

$ docker run --rm -it fedora:30 bash
$ dnf install -yq libatomic

$ ls -1 /usr/lib64 | grep libatomic
libatomic.so.1
libatomic.so.1.2.0

See that it is still version 1.2.0? And when linked there is nothing specific to the minor version, so it only depends upon the major version, linking to libatomic.so.1.

Regardless, that dep is already statically linked, Nuitka should not be attempting to link from MODLIBS in the first place when you already plan to link libpython.so instead of libpython.a (which is not present).

I don't think I want people that have their resulting program not working claim it's a Nuitka bug what these differences may do to their programs.

You're not responsible for users build environments.

Often developers run into issues building projects when they're inexperienced and raise issues on a project when the problem is their build environment rather than anything specific to the project.

Runtime concerns would be more valid.

Nuitka's current support for Zig as a compiler is iffy however. I've had to workaround that by adding a wrapper script as a proxy to reconstruct the command from Nuitka in a compatible way. Not all of it is Nuitka's fault though, I'll sort out a bug report for you later today to help address the Nuitka side for the 2.8 release.

but using newest Fedora is inherently going to make it harder to get stuff that works on other systems as well.

I was only sharing my own examples with Fedora, not that anyone would need that. Perhaps you're thinking about glibc as a compatibility issue, which is nothing to do with Fedora specifically, but any modern distro release.

As a side note: For portable binaries you want to build on old systems btw.
I am using a Linux container image tool for my commercial users that I mean to make public, but lack the time

python-build-standalone is doing the same with very old containers for their build environment.

In the past that was often the right approach, but these days one can also opt to use Zig (which I've made work with Nuitka via some workarounds).

Typically the only issue being resolved by the older build systems is a lower glibc. Zig takes care of that, removing a bunch of hassle that you can run into with aging build environments when this is the only real concern you're addressing.

If I am mistaken here, I'd appreciate knowing what else is a problem that only an old build environment would support.

So I think a far better option for Nuitka would be to properly support Zig as a backend, which is easily available through the ziglang python package? (it should not be a hard dependency however, one should be able to use Nuitka without that extra weight if they're providing Zig from elsewhere, such as Fedora's own package, or official builds retrieved from Zig directly)

Reference

Let's look at Python / libpython installed from UV and some distro packages.

python-build-standalone (provided Python via UV):

$ dnf install -yq uv patchelf
$ uv python install 3.13

$ patchelf --print-needed /root/.local/share/uv/python/cpython-3.13.6-linux-x86_64-gnu/bin/python3.13
libpthread.so.0
libdl.so.2
libutil.so.1
librt.so.1
libm.so.6
libc.so.6

$ patchelf --print-needed /root/.local/share/uv/python/cpython-3.13.6-linux-x86_64-gnu/lib/libpython3.13.so.1.0
libdl.so.2
librt.so.1
libm.so.6
libpthread.so.0
libutil.so.1
libc.so.6
ld-linux-x86-64.so.2

# Static linked libpython:
$ du -D --apparent-size --si /root/.local/share/uv/python/cpython-3.13.6-linux-x86_64-gnu/bin/python3.13
30M     /root/.local/share/uv/python/cpython-3.13.6-linux-x86_64-gnu/bin/python3.13

$ du -D --apparent-size --si /root/.local/share/uv/python/cpython-3.13.6-linux-x86_64-gnu/lib/libpython3.13.so.1.0
32M     /root/.local/share/uv/python/cpython-3.13.6-linux-x86_64-gnu/lib/libpython3.13.so.1.0

Fedora:

$ docker run --rm -it fedora:42 bash
# libpython comes via implicit dep `python3-libs`
$ dnf install -yq python patchelf which

$ ldd $(which python)
        linux-vdso.so.1 (0x00007ffea68ac000)
        libpython3.13.so.1.0 => /lib64/libpython3.13.so.1.0 (0x000076c64de03000)
        libc.so.6 => /lib64/libc.so.6 (0x000076c64dc11000)
        libm.so.6 => /lib64/libm.so.6 (0x000076c64db23000)
        /lib64/ld-linux-x86-64.so.2 (0x000076c64e3d2000)

$ patchelf --print-needed /usr/bin/python3
libpython3.13.so.1.0
libc.so.6

$ patchelf --print-needed /lib64/libpython3.13.so.1.0
libm.so.6
libc.so.6

# Very lightweight as libpython is dynamically linked:
$ du -D --apparent-size --si /usr/bin/python3
12k     /usr/bin/python3

$ du -D --apparent-size --si /usr/lib64/libpython3.13.so.1.0
5.7M    /usr/lib64/libpython3.13.so.1.0

NOTE: The remaining libs additionally linked by python-build-standalone will be present on glibc distros:

$ ls -1 /usr/lib64 | grep -E 'lib(pthread|rt|util)'
libpthread.so.0
librt.so.1
libutil.so.1

# These extra libs are all provided via the glibc package:
$ dnf provides --repo fedora '*/librt.so.1' '*/libutil.so.1' '*/libpthread.so.0' | grep -B3 ': /usr/lib64'

glibc-2.41-1.fc42.x86_64 : The GNU libc libraries
Repo         : fedora
Matched From :
Filename     : /usr/lib64/librt.so.1
--
glibc-2.41-1.fc42.x86_64 : The GNU libc libraries
Repo         : fedora
Matched From :
Filename     : /usr/lib64/libutil.so.1
--
glibc-2.41-1.fc42.x86_64 : The GNU libc libraries
Repo         : fedora
Matched From :
Filename     : /usr/lib64/libpthread.so.0

Ubuntu:

docker run --rm -it --env DEBIAN_FRONTEND=noninteractive ubuntu:24.04 bash

# Already established these are common libs bundled with glibc:
$ ls -1 /lib/x86_64-linux-gnu | grep -E 'lib(pthread|rt|util)'
libpthread.so.0
librt.so.1
libutil.so.1

# `libpython3.12` provides the shared library,
# it must be added explicitly as even `python3` package only brings in `libpython3.12-minimal`
$ apt-get -qq update && apt-get -qq install python3-minimal libpython3.12 patchelf

# Ubuntu's Python doesn't dynamically link to `libpython` (statically bundled):
# (`libexpat1` package was installed implicitly by the python package)
$ ldd $(which python3)
        linux-vdso.so.1 (0x00007ffef176d000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007b592dfb0000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007b592df94000)
        libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007b592df68000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007b592dd56000)
        /lib64/ld-linux-x86-64.so.2 (0x00007b592e09d000)

$ patchelf --print-needed $(which python3)
libm.so.6
libz.so.1
libexpat.so.1
libc.so.6

# Similar dep tree:
$ patchelf --print-needed /usr/lib/x86_64-linux-gnu/libpython3.12.so.1.0
libm.so.6
libz.so.1
libexpat.so.1
libc.so.6
ld-linux-x86-64.so.2

# libpython was statically linked :
$ du -D --apparent-size --si /usr/bin/python3
8.1M    /usr/bin/python3

# If wanting the static library available:
$ apt-get -qq install libpython3.12-dev
$ ls /lib/x86_64-linux-gnu | grep python
libpython3.12.a
libpython3.12.so
libpython3.12.so.1
libpython3.12.so.1.0

@kayhayen

I haven't gotten a chance to read through your messages in your entirety yet but I believe @kayhayen's analysis is broadly accurate and we do not yet provide libraries that are useful for Nuitka. For good reasons Nuitka would like a static libpython.a, but as you note, that library isn't present in the downloads that uv installs.

Is there a specific problem you're trying to solve @polarathene (e.g. are you trying to build something with Nuitka + uv's Python build) or are you just curious about the topic? I think the question I posed in this issue has been answered.

Is there a specific problem you're trying to solve @polarathene (e.g. are you trying to build something with Nuitka + uv's Python build) or are you just curious about the topic?

I had encountered this issue and was seeking clarity, which I've accomplished AFAIK :)

I've since shared my findings with both Nuitka and python-build-standalone project maintainers to have that resolved.
Providing libatomic.a is simple enough, I just wanted to assist with resolving it at both ends :)
Zig does not seem to like the -l:libatomic.a syntax, but it will accept -Bstatic -l atomic -Bdynamic.

As for a real-world project I was using OctoDNS to test with PyInstaller and Nuitka, followed by trying uv's provided Python instead of the system package (hitting this libatomic.a issue with Nuitka) and then sussing out cross-compilation via Zig (where uv will retrieve libpython.so for foreign archs as well).

No real problem presently, just getting familiar with tooling options 👍

For good reasons Nuitka would like a static libpython.a, but as you note, that library isn't present in the downloads that uv installs.

I only recently learned that uv can download python for foreign archs too, I had resorted to pulling from GH releases instead. Although that presently has some caveats, for an automated build environment it's easy enough to workaround in the meantime 😅

I would assume python-build-standalone could produce libpython.a and include that in the same GH releases that uv pulls from? You'd just add it as an opt-in option like --with-static-lib to uv python install, similar to how there is opt-out --no-bin to skip the python binary.

I also recall seeing a recent change to link musl dynamically to musl libc, making that less portable but I understand it was done so that it became possible to leverage dlopen() calls.

Navigation Menu

Repository navigation

What is needed in uv's Python to allow static libpython build? #3331

`python-build-standalone` prioritizes runtime compatibility over performance and size

`libatomic.a` shouldn't be an issue

`libpython.a` should only be present via opt-in

Unlikely to make a difference with manipulation concern?

`libatomic.a` is not needed

Investigation

Why is this linking logic triggered?

Proper approach

`libatomic.so` vs `libatomic.a` availability in packages

Inspecting the lib discovery process (link-time vs run-time)

Static or Dynamic?

Full response

Reference

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	# For NetBSD the rpath is required, on FreeBSD it's warned as unused, macOS
	# and AIX don't have it, but some Python flavors like Standalone Python
	# won't find it unless provided like this, so lets try to add this by
	# default and remove OSes that give errors one by one.
	if not isFreeBSD() and not isMacOS() and not isAIX():
	env.Append(LINKFLAGS=["-Wl,-rpath=" + python_lib_path])

	# Set load libpython from binary directory default
	if (
	env.gcc_mode
	and not isMacOS()
	and not isAIX()
	and not os.name == "nt"
	and (env.exe_mode or env.dll_mode)
	):
	# TODO: It would appear that this should be merged with above code that does
	# env.Append(LINKFLAGS=["-Wl,-rpath=" + python_lib_path]) relatively
	# unconditionally already.
	if env.standalone_mode:
	rpath = "$$ORIGIN"
	else:
	rpath = python_lib_path

	env.Append(LINKFLAGS=["-Wl,-R,'%s'" % rpath])

	# The rpath is no longer used unless we do this on modern Linux. The
	# option name is not very revealing, but basically without this, the
	# rpath in the binary will be ignored by the loader.
	# spell-checker: ignore dtags
	if isLinux():
	env.Append(LINKFLAGS=["-Wl,--disable-new-dtags"])

	# Requested or user provided, detect if it's clang even from environment
	if isClangName(env.the_cc_name):
	clang_mode = True
	env["CCVERSION"] = None

	# We consider clang to be a form of gcc for the most things, they strive to
	# be compatible.
	env.gcc_mode = isGccName(env.the_cc_name) or clang_mode
	env.clang_mode = clang_mode

	# Only use MSVC if not already clear, we are using MinGW.
	env.msvc_mode = os.name == "nt" and not env.gcc_mode
	env.mingw_mode = os.name == "nt" and env.gcc_mode
	env.clangcl_mode = clangcl_mode

	# After dependency detection, we can change the RPATH for main binary.
	if isRPathUsingPlatform():
	setSharedLibraryRPATH(
	os.path.join(dist_dir, standalone_entry_points[0].dest_path), "$ORIGIN"
	)

	def getModuleLinkerLibs():
	"""Get static link libraries needed."""
	try:
	import sysconfig
	except ImportError:
	return []
	else:
	# static link libraries might be there, spell-checker: ignore modlibs
	result = sysconfig.get_config_var("MODLIBS") or ""
	result = [entry[2:] for entry in result.split() if entry.startswith("-l:")]

	return result

	link_module_libs = getModuleLinkerLibs()
	if link_module_libs:
	scons_options["link_module_libs"] = ",".join(link_module_libs)

	# Any module libs that are for self-compiled Python to be static.
	env.Append(_LIBFLAGS=["-l" + lib_desc for lib_desc in link_module_libs])

What is needed in uv's Python to allow static libpython build? #3331

Description

Activity

kayhayen commented on Feb 7, 2025

kayhayen commented on May 11, 2025

WilliamStam commented on May 14, 2025

KRRT7 commented on May 14, 2025

kayhayen commented on Sep 12, 2025

polarathene commented on Oct 11, 2025

python-build-standalone prioritizes runtime compatibility over performance and size

libatomic.a shouldn't be an issue

libpython.a should only be present via opt-in

Unlikely to make a difference with manipulation concern?

libatomic.a is not needed

Investigation

Why is this linking logic triggered?

polarathene commented on Oct 11, 2025

Proper approach

libatomic.so vs libatomic.a availability in packages

Inspecting the lib discovery process (link-time vs run-time)

Static or Dynamic?

kayhayen commented on Oct 11, 2025

polarathene commented on Oct 12, 2025

Full response

Reference

geofft commented on Oct 12, 2025

polarathene commented on Oct 12, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`python-build-standalone` prioritizes runtime compatibility over performance and size

`libatomic.a` shouldn't be an issue

`libpython.a` should only be present via opt-in

`libatomic.a` is not needed

`libatomic.so` vs `libatomic.a` availability in packages