FFM API code base by DigitalSmile · Pull Request #458 · Pi4J/pi4j

DigitalSmile · 2025-03-29T20:08:53Z

This PR is work in progress, will be adding new implementations step by step.

Implementation details

FFM API provides new robust access to native code from JVM world. Moreover, all pi4j plugins relies on third party native libraries, which in the end calls linux kernel functions.
So, the typical flow was:
Java method (pi4j) -> JNA Java Code (jna) -> JNI Wrapper Call (pi4j) -> Native Library Call (gpiod, pigpio, etc) -> Kernel syscall
Now with the power of FFM API:
Java method (pi4j) -> Kernel syscall
This changes leads to better maintanability, less unsecure native calls, speed and better understanding of the code.

Main package

New plugin consists of several packages:

common package with internal structures needed to FFM work
- file - all needed code to access file descriptors with open/read/write/close/... methods
- gpio - all needed structures to work with GPIO (DigitalInput/Output) interface
- i2c - all needed structures to make I2C work (direct/ioctl/file). The only runtime dependency to work is i2c-tools
- ioctl - syscall wrapper to work with linux kernel HAL
- poll - syscall wrapper to work with poll/epoll messages
- serial - all needed structures to work with Serial interface
- spi - all needed structures to work with SPI bus
providers package with all user-facing implementations
- gpio - contains DigitalInputFFM and DigitalOutputFFM implementations
- i2c - contains I2CSMBus (using i2c-tools userspace library), I2CFile (using simple files) and I2CDirect (using ioctl)
- pwm - contains PwmFFMHardware implementation
- serial - contains SerialFFM implementation (beta version for now)
- spi - contains SpiFFM implementation

Tests

Since testing hardware is non-trivial tasks, I used an approach with different tests for different purposes.
Tests are organized in several packages:

unit - classic unit testing with support of Mockito framework. Mocks are located at package mocks
model - tests, that covers model conversion from Java objects to FFM API structures
integration - tests that are using mock drivers on linux kernel side to simulate the behaviour of real hardware.
- uses i2c-stub, gpio-sim and custom spi-mock kernel drivers (source code located in native folder)
- the main sequence: setup drivers -> ensure all drivers are active -> make echo calls with pi4j -> test the return result
- disabled for now, need to be moved to pi4j-harness project
jmh - classic JMH tests to measure speed of end-to-end calls with FFM/non-FFM plugins.

JMH Test results

Pi4j has 3 providers that are used in common:

gpiod provider with access to GPIO pins, uses libgpiod under the hood
pigpio provider with GPIO, I2C, PWM, Serial and SPI, uses libpigpio under the hood with access through /dev/mem
linuxfs provider with GPIO, I2C, PWM, Serial and SP, uses sysfs under the hood

All tests were run in VirtualBox environment using mock linux drivers:

GPIO using sim-gpio driver
I2C using i2c-stub driver
SPI using custom self made echo driver
Setup scripts are located here.

Because we are using mock drivers, we cannot test pigpio as it requires real hardware with available DMA addresses, so this library is out of scope of performance benchmark.
PWM is excluded from performance tests, because it is relying on filesystem in every provider available.
Serial is excluded, because FFM implementation is in beta and is out of scope with this PR.

How to read results

The results should be analysed in terms of relative numbers, but not absolute. The less is average time - the best score it wins.

GPIO (tested September, 20 2025)

OpenJDK Runtime Environment Corretto-25.0.0.36.2 (build 25+36-LTS)
Linux ds-VirtualBox 6.14.0-29-generic #29~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Aug 14 16:52:50 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
AMD Ryzen 7 5800X 8-Core Processor 29GiB System memory
Results:

Benchmark                                                  Mode  Cnt     Score    Error  Units
GPIOPerformanceTest.testFFMInputRoundTrip                  avgt    5     0.173 ±  0.003  ms/op
GPIOPerformanceTest.testGpioDInputRoundTrip                avgt    5    10.537 ±  6.196  ms/op
GPIOPerformanceTest.testLinuxFsInputRoundTrip              avgt    5   111.781 ± 42.199  ms/op

Benchmark                                                  Mode  Cnt     Score    Error  Units
GPIOPerformanceTest.testFFMInputWithListenerRoundTrip      avgt    5     1.594 ±  5.830  ms/op
GPIOPerformanceTest.testGpioDInputWithListenerRoundTrip    avgt    5     9.015 ± 12.139  ms/op
GPIOPerformanceTest.testLinuxFsInputWithListenerRoundTrip  avgt    5   110.115 ± 18.796  ms/op

Benchmark                                                  Mode  Cnt     Score    Error  Units
GPIOPerformanceTest.testFFMOutputRoundTrip                 avgt    5     0.042 ±  0.001  ms/op
GPIOPerformanceTest.testGpioDOutputRoundTrip               avgt    5     0.110 ±  0.002  ms/op
GPIOPerformanceTest.testLinuxFsOutputRoundTrip             avgt    5   108.306 ± 12.490  ms/op

SPI (tested September, 20 2025)

OpenJDK Runtime Environment Corretto-25.0.0.36.2 (build 25+36-LTS)
Linux ds-VirtualBox 6.14.0-29-generic #29~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Aug 14 16:52:50 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
AMD Ryzen 7 5800X 8-Core Processor 29GiB System memory
Results:

Benchmark                                         Mode  Cnt  Score   Error  Units
SPIPerformanceTest.testFFMWriteReadRoundTrip      avgt    5  0.044 ± 0.006  ms/op
SPIPerformanceTest.testLinuxFsWriteReadRoundTrip  avgt    5  0.076 ± 0.001  ms/op

I2C (tested September, 20 2025)

OpenJDK Runtime Environment Corretto-25.0.0.36.2 (build 25+36-LTS)
Linux ds-VirtualBox 6.14.0-29-generic #29~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Aug 14 16:52:50 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
AMD Ryzen 7 5800X 8-Core Processor 29GiB System memory
Results:

Benchmark                                 Mode  Cnt  Score   Error  Units
I2CPerformanceTest.testFFMSMBusRoundTrip  avgt    5  0.038 ± 0.002  ms/op

…nce testing

DigitalSmile · 2025-03-31T06:43:45Z

For the history, benchmark for the simple roundtrip (create, get state, shutdown):

Result "com.pi4j.plugin.jmh.DigitalInputPerformanceTest.testCreateShutdownRoundTrip":
  1.652 ±(99.9%) 0.121 ms/op [Average]
  (min, avg, max) = (1.615, 1.652, 1.698), stdev = 0.031
  CI (99.9%): [1.531, 1.772] (assumes normal distribution)

eitch · 2025-03-31T13:13:14Z

For the history, benchmark for the simple roundtrip (create, get state, shutdown):

Result "com.pi4j.plugin.jmh.DigitalInputPerformanceTest.testCreateShutdownRoundTrip":
  1.652 ±(99.9%) 0.121 ms/op [Average]
  (min, avg, max) = (1.615, 1.652, 1.698), stdev = 0.031
  CI (99.9%): [1.531, 1.772] (assumes normal distribution)

Perhaps you can add a benchmark for the existing provider?

DigitalSmile · 2025-03-31T13:52:45Z

For the history, benchmark for the simple roundtrip (create, get state, shutdown):
Result "com.pi4j.plugin.jmh.DigitalInputPerformanceTest.testCreateShutdownRoundTrip":
  1.652 ±(99.9%) 0.121 ms/op [Average]
  (min, avg, max) = (1.615, 1.652, 1.698), stdev = 0.031
  CI (99.9%): [1.531, 1.772] (assumes normal distribution)
Perhaps you can add a benchmark for the existing provider?

That's for the the stage 2 :) I tried to quickly do it and failed, due to lack of build targets for desktop (amd64) of natives. I can try to run the benchmark on a real hardware, but since we are testing in comparison with other approaches, it is better to stay with gpio-sim.
Anyway will try to build locally later on.

DigitalSmile · 2025-04-01T18:22:52Z

Okay, a bit of journey on benchmarks :)

TLDR

For libgpiod (Java code -> JNA Code -> libpi4j-gpiod.so (JNI-wrapper) -> libgpiod.so (native library) -> GPIO kernel syscalls)

Result "com.pi4j.plugin.gpiod.jmh.DigitalInputPerformanceTest.testCreateShutdownRoundTrip":
  0.252 ±(99.9%) 0.015 ms/op [Average]
  (min, avg, max) = (0.248, 0.252, 0.257), stdev = 0.004
  CI (99.9%): [0.236, 0.267] (assumes normal distribution)

For FFM API (Java code -> GPIO kernel syscalls)

Result "com.pi4j.plugin.ffm.jmh.DigitalInputPerformanceTest.testCreateShutdownRoundTrip":
  0.167 ±(99.9%) 0.008 ms/op [Average]
  (min, avg, max) = (0.165, 0.167, 0.170), stdev = 0.002
  CI (99.9%): [0.159, 0.175] (assumes normal distribution)

Bit of details

So first things first I built manually all gpiod natives (that was a lot more easier without maven) and pointed to the library folder in the code of NativeLibraryLoader class. Then I had to comment out lines in GpioDContext to prevent failing since I am not running Pi (btw, why there is such a limitation?).
My naive approach was to write the similar test like in https://github.com/Pi4J/pi4j/blob/e1f3079e19c0025271cb7bb947b6e05023e86038/plugins/pi4j-plugin-ffm/src/test/java/com/pi4j/plugin/jmh/DigitalInputPerformanceTest.java The problems begun immediately with SIGSEV from JVM, indicating that memory is corrupted.
malloc(): unsorted double linked list corrupted
I digged and found out, that threads created by monitorLineEvents are usually freeze and unable to shutdown properly. It seems the heap goes crazy in between native calls <-> monitoring thread <-> main java code. When I commented out the monitoring submission to ExecutorService all benchmarks went fine giving me above result.

So for the sake of comparison I did the same thing with FFM code (commented out monitoring thread creation) and rerun benchmark.

The results are as expected almost similar, with a slight faster with FFM. That happens, just because the roundtrip between native world and Java world is shorter.
Anyway, all above is jfyi, so I don't think we need to take any action items on that. I will continue with feature implementations. Next is i2c.

eitch · 2025-04-02T06:56:23Z

Nice write up! Thanks a lot for that insight!

taartspi · 2025-04-03T03:11:32Z

plugins/pi4j-plugin-ffm/src/main/java/com/pi4j/plugin/ffm/providers/spi/FFMSpi.java

+        // if 'flags' were provided in the SPI config, then accept them
+        if(config().flags() != null){
+            flags = config().flags().intValue();
+        }


in the linuxfs spi implementation we error out is the flags parm was used, This pertained to the pigpio provider and it continued use would put limits on the linuxfs code.

taartspi · 2025-04-03T03:13:48Z

plugins/pi4j-plugin-ffm/src/main/java/com/pi4j/plugin/ffm/providers/spi/FFMSpi.java

+        if(bus.getBus() > 1){
+            throw new IOException("Unsupported BUS by PiGPIO SPI Provider: bus=" + bus.toString());
+        }
+


also, ignoring the old pigpio limitations, the linuxfs spi will allow any bus and channel the user wants. They are required to do the config.txt updates to the spi actually exists.

taartspi · 2025-04-03T03:14:40Z

plugins/pi4j-plugin-ffm/src/main/java/com/pi4j/plugin/ffm/providers/spi/FFMSpi.java

+        if(bus == SpiBus.BUS_1 && (mode == SpiMode.MODE_1 || mode == SpiMode.MODE_3)) {
+            throw new IOException("Unsupported SPI mode on AUX SPI BUS_1: mode=" + mode.toString());
+        }
+


broken record, this limitation does not exist in linuxfs

DigitalSmile · 2025-04-03T07:04:32Z

@taartspi just for the record - SPI classes are just the boilerplate code now, I copy-pasted it for future reference. You can have a look on my implementation here (which I'll be using): https://github.com/DigitalSmile/gpio/blob/main/src/main/java/org/digitalsmile/gpio/spi/SPIBus.java

…nce testing

# Conflicts: # pi4j-core/src/main/java/com/pi4j/io/pwm/PwmBase.java

# Conflicts: # plugins/pi4j-plugin-linuxfs/src/main/java/com/pi4j/plugin/linuxfs/LinuxFsPlugin.java # plugins/pi4j-plugin-linuxfs/src/test/java/com/pi4j/plugin/linuxfs/PWMChipTest.java

plugins/pi4j-plugin-ffm/src/main/java/com/pi4j/plugin/ffm/providers/pwm/PwmFFMProviderImpl.java

taartspi · 2025-09-23T18:43:29Z

Looks the merge is complete and ffm updates are ready for HW testing

taartspi

I have reviewed parts of the code not all. This delivery is a great deal of work. I will continue to review and test I think we should merge and build this to make testing and any new issue discussions easier.

eitch · 2025-09-24T07:15:59Z

So i'll update develop to 4.0.0-SNAPSHOT and then merge this branch.

Any objections? @FDelporte @DigitalSmile @taartspi

FDelporte · 2025-09-24T07:21:15Z

@eitch the branch is already on <version>4.0.0-SNAPSHOT</version>.
So yes, please, let's merge and speed up testing 🚀
There is only one direction: FORWARD 😉

eitch · 2025-09-24T07:22:52Z

Done.

DigitalSmile added 2 commits March 26, 2025 20:34

#454 Initial commit with some boilerplate

d748dd0

#454 Implemented DigitalInput, test setup and some unit tests

7647dd0

DigitalSmile requested review from FDelporte, dariuszzbyrad, eitch and taartspi March 29, 2025 20:09

DigitalSmile self-assigned this Mar 29, 2025

#454 Implemented DigitalOutput, some unit tests, initial JMH Performa…

e1f3079

…nce testing

#454 Test commit of generated sources

5d5a1ed

taartspi reviewed Apr 3, 2025

View reviewed changes

DigitalSmile and others added 12 commits April 10, 2025 11:16

#454 Add explicit FFM structures, implemented direct and smbus i2c

a6364cb

Testing pi4j-builder user

51b1d8d

Testing pi4j-builder user

b8b30d1

Update pi4j-builder user for docker run

50e9f3f

Update ci-pull-request.yml

f0949a2

Update ci-pull-request.yml

f88150f

Update ci-pull-request.yml

91a5176

Update ci-pull-request.yml

3b38b82

#454 Initial commit with some boilerplate

f069a12

#454 Implemented DigitalInput, test setup and some unit tests

4be0231

#454 Implemented DigitalOutput, some unit tests, initial JMH Performa…

4b92763

…nce testing

#454 Test commit of generated sources

d20c809

FDelporte and others added 18 commits September 10, 2025 16:07

Fix merge conflicts

1246655

Fix JavaDoc

3d98417

#454 Fix test

775956f

#454 Fix pom file to add ffm-plugin

cc4c5e8

Merge branch 'develop' into ffm-api

c7fd83c

# Conflicts: # pi4j-core/src/main/java/com/pi4j/io/pwm/PwmBase.java

Fix merge error

11d5d45

Merge branch 'develop' into ffm-api

848f94a

Align LinuxFS PWM with develop (merge error?)

9b82d63

#454 Fix typo in smbus library method

56a388e

#454 Fix aarch64 library path

6e27392

#454 Fix shutdown sequence for PWM

7efb346

#454 Fix PWM duty cycle type

b509551

#454 Added gpiod and linuxfs JMH tests

c2ec8c9

Change provider priorities

dea60aa

#454 Added PermissionHelper instance to check user/devices, fix tests

a079987

#454 Fix tests for build

2fa04fe

Merge remote-tracking branch 'origin/develop' into ffm-api

fa85669

# Conflicts: # plugins/pi4j-plugin-linuxfs/src/main/java/com/pi4j/plugin/linuxfs/LinuxFsPlugin.java # plugins/pi4j-plugin-linuxfs/src/test/java/com/pi4j/plugin/linuxfs/PWMChipTest.java

Align ID and Name of providers with others

8c4b782

taartspi requested changes Sep 22, 2025

View reviewed changes

plugins/pi4j-plugin-ffm/src/main/java/com/pi4j/plugin/ffm/providers/pwm/PwmFFMProviderImpl.java Show resolved Hide resolved

FDelporte added 3 commits September 22, 2025 17:47

Align ID and Name of providers with others, fix typo

f3a1e32

Merge branch 'develop' into ffm-api

b4e8ca3

Merge branch 'develop' into ffm-api

8f04cfe

DigitalSmile requested a review from taartspi September 23, 2025 16:12

taartspi reviewed Sep 23, 2025

View reviewed changes

eitch merged commit b18532e into develop Sep 24, 2025
1 check passed

eitch deleted the ffm-api branch September 24, 2025 07:22

Conversation

DigitalSmile commented Mar 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Implementation details

Main package

Tests

JMH Test results

How to read results

GPIO (tested September, 20 2025)

SPI (tested September, 20 2025)

I2C (tested September, 20 2025)

Uh oh!

DigitalSmile commented Mar 31, 2025

Uh oh!

eitch commented Mar 31, 2025

Uh oh!

DigitalSmile commented Mar 31, 2025

Uh oh!

DigitalSmile commented Apr 1, 2025

TLDR

Bit of details

Uh oh!

eitch commented Apr 2, 2025

Uh oh!

taartspi Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

taartspi Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

taartspi Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

DigitalSmile commented Apr 3, 2025

Uh oh!

Uh oh!

taartspi commented Sep 23, 2025

Uh oh!

taartspi left a comment

Choose a reason for hiding this comment

Uh oh!

eitch commented Sep 24, 2025

Uh oh!

FDelporte commented Sep 24, 2025

Uh oh!

Uh oh!

eitch commented Sep 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

DigitalSmile commented Mar 29, 2025 •

edited

Loading