Throughout my education and my career I have largely avoided native development and concentrated on developing in Java, recently however I have been working with a lot more native code so now it is time to come back and look at the relationship between these two worlds.
My learning and development style has always been to build from small working examples and iteratively expand, by iterating when you run into problems hopefully the delta between the current state and the last working state is not too large making it easier to diagnose the issues.
This blog post describes the steps I have taken to first create a native library and then to look at how this can be invoked using JNI or the newer Foreign Function and Memory API.
The example code can be found in my java-native-experiments GitHub repository, I would recommend using the beginning-native-java-development branch as after publishing this blog post the main branch may diverge further.
The Shared Library
The first step was to develop a shared library that will be called, in general I believe engineers would be looking at invoking native code either when a need to implement natively is identified or if a library already exists with the desired functionality.
In this project I am creating a new library called simple-library providing the API in the simple-library.h header.
int add_one(int x);
void say_hello();
The implementation is as simple as the names suggest:
int add_one(int x)
{
return x + 2;
}
void say_hello()
{
printf("Hello, from simple-library!\n");
}
I believe writing Makefiles is simple but as I have been using it for other projects I have used CMake and the following CMakeLists.txt to generate this for me:
cmake_minimum_required(VERSION 3.10.0)
if (EXISTS $ENV{HOME}/local)
set(CMAKE_INSTALL_PREFIX $ENV{HOME}/local)
message("Setting CMAKE_INSTALL_PREFIX to $ENV{HOME}/local")
endif()
project(simple-library VERSION 0.1.0 LANGUAGES C)
set(SIMPLE_SOURCE
simple-library.c
)
set (SIMPLE_HEADERS
simple-library.h
)
# Build the shared library
add_library(simple-library SHARED
${SIMPLE_SOURCE})
# Build the static library
add_library(simple-library-static STATIC
${SIMPLE_SOURCE})
# Install the shared library
install(TARGETS simple-library
LIBRARY DESTINATION lib)
# Install the static library
install(TARGETS simple-library-static
ARCHIVE DESTINATION lib)
# Install the header file
install(FILES ${SIMPLE_HEADERS}
DESTINATION include)
At the base of my home area I have a directory called local and running make install will install the libraries under $HOME/local/lib and the header file for the API under $HOME/local/include. For this example I only need a shared library but I have also been experimenting with a static library so both variants are installed.
The Native Application
The purpose of this investigation was to look at the pieces need to invoke the library from Java but as I said in the beginning I like to incrementally iterate so my next step was a simple C program to call the library. If it is not possible to call the library from a C application I would not be ready to try and call it from Java.
The following is the implementation of the simple-c-app:
#include <stdio.h>
#include "simple-library.h"
int main(int, char**){
printf("Hello, from simple-c-app!\n");
int a = 5;
printf("add_one(%d) = %d\n", a, add_one(a));
say_hello();
}
As with the library I also use CMake to generate the Makefile:
cmake_minimum_required(VERSION 3.10.0)
project(simple-c-app VERSION 0.1.0 LANGUAGES C)
add_executable(simple-c-app main.c)
target_include_directories(simple-c-app PRIVATE
$ENV{HOME}/local/include)
# TODO This seems to work for now but should it reference
# using find_library instead?
target_link_directories(simple-c-app PRIVATE
$ENV{HOME}/local/lib)
# Link the shared library
target_link_libraries(simple-c-app PRIVATE
simple-library)
# Link the static library
#target_link_libraries(simple-c-app PRIVATE
# simple-library-static)
As you can see at the bottom I have also been experimenting by being able to switch between the shared and the static library.
After building the app it can be invoked to check the interactions with the shared library:
$ build/simple-c-app
Hello, from simple-c-app!
add_one(5) = 6
Hello, from simple-library!
The Java JNI Application
For the Java JNI application we need to begin our development in Java to define the native methods from the perspective of Java and generate a header file for these methods, we then need to implement a shared library that implements the functions in the header file and call the target shared library before we can come back and run the Java application. In a project it may be more likely to develop a Java library for the native invocations separately from the main application. For the Java development I am currently using the latest Java 24 Temurin build.
The simple-jni Java project contains a single App class:
public class App {
/*
* Native Methods.
*/
private static native void sayHello();
private static native int addOne(final int x);
public static void main(String[] args) {
System.out.println("Java says Hello World!");
System.out.println(System.getProperty("java.library.path"));
System.loadLibrary("jni-library");
//System.loadLibrary("simple-library");
final int x = 11;
System.out.printf("addOne(%d)= %d\n", x, addOne(x));
sayHello();
System.out.println("Java says Goodbye World!");
}
}
This class defines two methods that we wish to invoke using JNI as well as the main method to invoke them. These methods do follow the same pattern as for the API in our shared library but this is not necessary, as you will see the JNI library will act as an intermediary so an alternative API could have been used.
We need the Java compiler to generate a header file for the native methods so the compiler plugin in the pom.xml is configured as:
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.13.0</version>
<configuration>
<compilerArgs>
<arg>-h</arg>
<arg>target/include</arg>
</compilerArgs>
</configuration>
</plugin>
The -h target/include causes the compiler to generate the following header under target/include:
/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class dev_lofthouse_App */
#ifndef _Included_dev_lofthouse_App
#define _Included_dev_lofthouse_App
#ifdef __cplusplus
extern "C" {
#endif
/*
* Class: dev_lofthouse_App
* Method: sayHello
* Signature: ()V
*/
JNIEXPORT void JNICALL Java_dev_lofthouse_App_sayHello
(JNIEnv *, jclass);
/*
* Class: dev_lofthouse_App
* Method: addOne
* Signature: (I)I
*/
JNIEXPORT jint JNICALL Java_dev_lofthouse_App_addOne
(JNIEnv *, jclass, jint);
#ifdef __cplusplus
}
#endif
#endif
This brings us to the next project jni-library where the functions in this header will be implemented in jni-library.c:
#include <stdio.h>
#include "dev_lofthouse_App.h"
#include "simple-library.h"
JNIEXPORT void JNICALL Java_dev_lofthouse_App_sayHello (JNIEnv *, jclass)
{
say_hello();
}
JNIEXPORT jint JNICALL Java_dev_lofthouse_App_addOne (JNIEnv * env, jclass jcl, jint x)
{
return add_one(x);
}
As at the most in this example we are passing integers I can ignore the additional function parameters for now. As I did with the previous two native projects I used CMake to configure the Makefile:
cmake_minimum_required(VERSION 3.10.0)
if (EXISTS $ENV{HOME}/local)
set(CMAKE_INSTALL_PREFIX $ENV{HOME}/local)
message("Setting CMAKE_INSTALL_PREFIX to $ENV{HOME}/local")
endif()
project(jni-library VERSION 0.1.0 LANGUAGES C)
set(JNI_SOURCE
jni-library.c
)
add_library(jni-library SHARED ${JNI_SOURCE})
# Include the JNI headers
target_include_directories(jni-library PRIVATE
$ENV{JAVA_HOME}/include
$ENV{JAVA_HOME}/include/linux)
# Include the simple-jni headers
target_include_directories(jni-library PRIVATE
../simple-jni/target/include)
# Include the simple-library headers
target_include_directories(jni-library PRIVATE
$ENV{HOME}/local/include) # This one was installed.
# Link the shared library
target_link_directories(jni-library PRIVATE
$ENV{HOME}/local/lib)
# Link the shared library
target_link_libraries(jni-library PRIVATE
simple-library)
# Install the shared library
install(TARGETS jni-library
LIBRARY DESTINATION lib)
This library needs access to a few more headers than were needed for simple-c-app, obviously we need the headers for the shared library we will call but we also need the generated JNI header as well as the headers in the Java installation as well. Similar to the experiments with the simple-c-app I could have targeted the static library instead of building against the shared library.
Before coming back to the simple-jni project to run the code it is worth pointing out here that as the shared libraries are being installed in a non-standard location they may not be found correctly at runtime. As an example if we run the following command:
$ ldd ~/local/lib/libjni-library.so
linux-vdso.so.1 (0x00007a17d59ea000)
libsimple-library.so => not found
libc.so.6 => /usr/lib/libc.so.6 (0x00007a17d57d0000)
/usr/lib64/ld-linux-x86-64.so.2 (0x00007a17d59ec000)
The output shows that libsimple-library.so was not found.
However once the LD_LIBRARY_PATH environment variable is set the library can be found:
$ export LD_LIBRARY_PATH=$HOME/local/lib
$ ldd ~/local/lib/libjni-library.so
linux-vdso.so.1 (0x000070f866a98000)
libsimple-library.so => /home/darranl/local/lib/libsimple-library.so (0x000070f866a88000)
libc.so.6 => /usr/lib/libc.so.6 (0x000070f866879000)
/usr/lib64/ld-linux-x86-64.so.2 (0x000070f866a9a000)
Back in the simple-jni project we can now run the previously built Java command using the run-app.sh :
#!/bin/bash
export LD_LIBRARY_PATH=$HOME/local/lib
java --class-path=target/simple-jni-0.0.1-SNAPSHOT.jar \
--enable-native-access=ALL-UNNAMED \
dev.lofthouse.App
If only the JNI library needed to found the system property -Djava.library.path=$HOME/local/lib could be set instead. Running the application gives us the following output:
$ ./run-app.sh
Java says Hello World!
/home/darranl/local/lib:/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
addOne(11)= 12
Hello, from simple-library!
Java says Goodbye World!
The Foreign API Java Application
The final part of this investigation was in the simple-foreign project to use the new Foreign Function APIs, unlike the JNI example which required an intermediary JNI aware native library this Java application can call the original shared library directly.
The App class is more involved as the functions to be called need to be describes but with the eliminated round trips to develop and build a JNI library:
import static java.lang.foreign.ValueLayout.JAVA_INT;
import java.lang.foreign.Arena;
import java.lang.foreign.FunctionDescriptor;
import java.lang.foreign.Linker;
import java.lang.foreign.MemorySegment;
import java.lang.foreign.SymbolLookup;
import java.lang.invoke.MethodHandle;
/**
* Hello world!
*/
public class App {
public static void main(String[] args) throws Throwable {
Arena confinedArena = Arena.ofConfined();
Linker linker = Linker.nativeLinker();
SymbolLookup simpleLibraryLookup =
SymbolLookup.libraryLookup("libsimple-library.so", confinedArena);
/*
* The first function is the say_hello function which takes no arguments,
* returns void and prints a message to the output.
*/
// Find the reference to the say_hello function.
MemorySegment sayHelloSymbol = simpleLibraryLookup.find("say_hello").get();
// Describe the function signature, this function takes no arguments and returns void.
FunctionDescriptor sayHelloDescriptor = FunctionDescriptor.ofVoid();
// Convert to a MethodHandle for the function.
MethodHandle sayHelloMethodHandle = linker.downcallHandle(sayHelloSymbol, sayHelloDescriptor);
// Invoke the function.
sayHelloMethodHandle.invokeExact();
/*
* The second function is the add_one function which takes an integer as an argument,
* adds two to it and returns the result.
*/
// Find the reference to the add_one function.
MemorySegment addOneSymbol = simpleLibraryLookup.find("add_one").get();
// Describe the function signature, this function takes an integer and returns an integer.
FunctionDescriptor addOneDescriptor = FunctionDescriptor.of(JAVA_INT, JAVA_INT);
// Convert to a MethodHandle for the function.
MethodHandle addOneMethodHandle = linker.downcallHandle(addOneSymbol, addOneDescriptor);
// Invoke the function.
int result = (int) addOneMethodHandle.invokeExact(14);
// Display the result.
System.out.printf("addOne(%d) = %d\n", 14, result);
System.out.println("Hello World!");
}
}
It is worth noting that after the calls to set up access the end result is we have a java.lang.invoke.MethodHandle to use for the invocations which has been available for Java Reflection going back to Java 1.7.
Similar to the JNI example we need LD_LIBRARY_PATH to be set to our non standard location and we can use the run-app.sh script to call the class.
#!/bin/bash
export LD_LIBRARY_PATH=$HOME/local/lib
java --class-path=target/simple-foreign-0.0.1-SNAPSHOT.jar \
--enable-native-access=ALL-UNNAMED \
dev.lofthouse.App
This gives us the following output:
$ ./run-app.sh
Hello, from simple-library!
addOne(14) = 15
Hello World!
Conclusion
This concludes this first experiment to try both approaches to native invocations side by side.
Overall both approaches feel like they still need an individual with experience of both the native side of the calls and the Java side of the code to be able to develop the intermediary layer. In the case of JNI this was largely in the place of implementing the generated header for the native methods, in the foreign function case this was in the place of fully describing the the native functions first using Java code.
The examples here were very simple at the most passing ints, working with Java Objects, native Structs and calling back from the native code to the Java code would all add more complexity to test both approaches further.
Before looking at the more advanced examples the next step I would like to investigate and hopefully publish a follow up blog is updating these examples to be compiled to native code using GraalVM.
Darran's Development Blog 