Skip to content

Comments

external_deps: rebuild nacl_loader for Linux amd64#1638

Merged
slipher merged 1 commit intoDaemonEngine:masterfrom
slipher:build-naclloader
Apr 8, 2025
Merged

external_deps: rebuild nacl_loader for Linux amd64#1638
slipher merged 1 commit intoDaemonEngine:masterfrom
slipher:build-naclloader

Conversation

@slipher
Copy link
Member

@slipher slipher commented Apr 3, 2025

Add the naclruntime target to external_deps/build.sh. It builds nacl_loader and nacl_helper_bootstrap.
This uses our version of the native_client repository, with a hacked build script that lets us build native binaries without Chromium tools. The build brings two dependencies: LLVM and SCons. Though we already needed LLVM for the MSVC platforms, for the silly reason of generating .libs.

Fixes #1504 (NaCl crash handling doesn't work with recent Linux kernels.)

TODO:

  • Remove nacl_loader from the Linux amd64 naclsdk target so they don't step on each other.
  • Add naclruntime to the package lists.

@slipher
Copy link
Member Author

slipher commented Apr 3, 2025

One more thing I would like to do is see if I can run the tests. (Manually, not in the script.) Unvanquished seems to work, but there are native client tests that verify e.g. stuff about how the binary is linked, which could be important for security properties. I might attempt to run the tests by copying the built binaries into an un-hacked version of native_client that still uses Chromium tools.

@illwieckz
Copy link
Member

Not your fault, but this is annoying:

In file included from <built-in>:385:
<command line>:1:8: error: undefining builtin macro [-Werror,-Wbuiltin-macro-redefined]
    1 | #undef __STDC_HOSTED__
      |        ^

@illwieckz
Copy link
Member

Actually I'm very happy that you investigate rebuilding the loader!

@slipher
Copy link
Member Author

slipher commented Apr 3, 2025

Not your fault, but this is annoying:

What version of LLVM do you have?

Can you try running scons platform=x86-64 MODE=opt-host naclsdk_validate=0 sysinfo=0 sel_ldr from the native_client folder and post the log?

@illwieckz
Copy link
Member

What version of LLVM do you have?

My default clang is 18.1.3, does it use Clang or GCC?

Can you try running scons platform=x86-64 MODE=opt-host naclsdk_validate=0 sysinfo=0 sel_ldr from the native_client folder and post the log?

scons platform=x86-64 MODE=opt-host naclsdk_validate=0 sysinfo=0 sel_ldr from the native_client
site_init:132: SyntaxWarning: invalid escape sequence '\$'
scons: Reading SConscript files ...
Unvanquished/daemon/external_deps/build-linux-amd64-default_10/naclruntime/DaemonEngine-native_client-27ccec9/SConstruct:3628: SyntaxWarning: invalid escape sequence '\c'
  COVERAGE_ANALYZER='$COVERAGE_ANALYZER_DIR\coverage_analyzer.exe',
======================================================================
Warning: "--mode" did not specify both trusted and untrusted build environments.  As a result, many tests will not be run.
======================================================================
Unvanquished/daemon/external_deps/build-linux-amd64-default_10/naclruntime/DaemonEngine-native_client-27ccec9/src/trusted/service_runtime/build.scons:826: SyntaxWarning: invalid escape sequence '\['
  RE_IDENT = '^\[[0-9,:.]*\] (e_ident\+1 = ELF)$'
SKIPPING test  text_overlaps_rodata
SKIPPING test  text_overlaps_data
There are 1 broken tests. Add --verbose to the command line for more information.
======================================================================
B U I L D - O U T P U T:
======================================================================
scons: done reading SConscript files.
scons: Building targets ...
________Compiling scons-out/opt-linux-x86-64/obj/src/trusted/service_runtime/linux/nacl_bootstrap.o
In file included from <built-in>:385:
<command line>:1:8: error: undefining builtin macro [-Werror,-Wbuiltin-macro-redefined]
    1 | #undef __STDC_HOSTED__
      |        ^
1 error generated.
scons: *** [scons-out/opt-linux-x86-64/obj/src/trusted/service_runtime/linux/nacl_bootstrap.o] Error 1
scons: building terminated because of errors.

**********************************************************************
ERROR REPORT: 1 failures
**********************************************************************

scons-out/opt-linux-x86-64/obj/src/trusted/service_runtime/linux/nacl_bootstrap.o failed: Error 1

@slipher slipher force-pushed the build-naclloader branch from 53a5277 to a50a1e4 Compare April 4, 2025 00:27
"${download_only}" && return

cd "${dir_name}"
scons "platform=${NACL_ARCH}" MODE=opt-host werror=0 naclsdk_validate=0 sysinfo=0 sel_ldr
Copy link
Member

@illwieckz illwieckz Apr 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can do: MODE=opt-linux to be explicit.

Anyway it is not possible to cross-compile with the current Scons build.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Anyway it is not possible to cross-compile with the current Scons build.

Maybe if I hadn't commented out the sysroot part hehe

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A lot of code like that just detects the target by scanning the host (even GetOS() relies on Python's sys.platform…):

# Some of our tools utilize a unique platform string which is used to
# distinguish between platform and architectures.
def PlatformTriple(platform=None, machine=None):
  os = GetOS(platform)
  arch3264 = GetArch3264(machine)

  if os == OS_WIN:
    if IsCygWin(platform):
      return 'i686-pc-cygwin'
    else:
      return 'i686-w64-mingw32'
  elif os == OS_MAC:
    return 'x86_64-apple-darwin'
  elif os == OS_LINUX:
    if arch3264 == ARCH3264_ARM:
      # TODO(mcgrathr): How to distinguish gnueabi vs gnueabihf?
      return 'arm-linux-gnueabihf'
    elif arch3264 == ARCH3264_X86_32:
      return 'i686-linux'
    elif arch3264 == ARCH3264_X86_64:
      return 'x86_64-linux'

  raise Exception('Unknown platform and machine')

Copy link
Member Author

@slipher slipher Apr 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried with upstream native_client and it manages to build something for ARM if I do sudo apt install gcc-arm-linux-gnueabihf and ./scons --verbose platform=arm MODE=opt-linux naclsdk_validate=0 sysinfo=0 -j1 sel_ldr. (Yes, it now uses GCC instead of Clang...)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird, I get many other errors when I try that…

Copy link
Member

@illwieckz illwieckz Apr 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, here is the error I get when attempting to cross-compile an arm loader:

$ scons platform=arm MODE=opt-linux naclsdk_validate=0 werror=0 sysinfo=0 sel_ldr
scons: Reading SConscript files ...
======================================================================
Warning: "--mode" did not specify both trusted and untrusted build environments.  As a result, many tests will not be run.
======================================================================
*** SKIPPING  opt-linux-32 : run_nacl_desc_io_alloc_ctor_test
AttributeError: 'SConsEnvironment' object has no attribute 'Program':
  File "native_client/SConstruct", line 3887:
    BuildEnvironments(selected_envs)
  File "site_init", line 203:
    
  File "/usr/lib/python3/dist-packages/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "native_client/site_scons/site_tools/defer.py", line 148:
    func(env)
  File "site_init", line 125:
    
  File "/usr/lib/python3/dist-packages/SCons/Script/SConscript.py", line 598:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/usr/lib/python3/dist-packages/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "native_client/src/trusted/validator_arm/build.scons", line 271:
    nexe = untrusted_env.ComponentProgram(test, 'testdata/' + test + '.S',
  File "/usr/lib/python3/dist-packages/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "native_client/site_scons/site_tools/component_builders.py", line 485:
    out_nodes = env.Program(prog_name, *args, **kwargs)

@slipher slipher force-pushed the build-naclloader branch 2 times, most recently from a4f70d0 to 4d9cc75 Compare April 6, 2025 11:05
@slipher slipher marked this pull request as ready for review April 6, 2025 11:05
Copy link
Member

@illwieckz illwieckz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@illwieckz
Copy link
Member

What is the reason for not having cloned the full native_client repository? The size?

@illwieckz
Copy link
Member

What is the reason for not having cloned the full native_client repository? The size?

Right, mentionned in README:

History had to be cut off due to 200 MB zip files in the history.

@slipher slipher force-pushed the build-naclloader branch from 4d9cc75 to bfb4be2 Compare April 8, 2025 01:16
Add the naclruntime target to external_deps/build.sh. It builds
nacl_loader and nacl_helper_bootstrap.
This uses our version of the native_client repository with a hacked
build script that lets us build native binaries without Chromium tools.
The build brings two dependencies: LLVM and SCons.

Fixes DaemonEngine#1504 (NaCl crash handling doesn't work with recent Linux
kernels.)
@slipher slipher force-pushed the build-naclloader branch from bfb4be2 to 0353e12 Compare April 8, 2025 01:17
@slipher slipher merged commit bc09529 into DaemonEngine:master Apr 8, 2025
4 checks passed
@slipher slipher deleted the build-naclloader branch April 8, 2025 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NaCl amd64 crash dumps no longer work on Linux

2 participants