pkg/xipfs: add XIPFS as vfs module#21569
Conversation
GGuche-2XS-Univ-Lille
requested review from
MichelRottleuthner,
aabadie,
jia200x and
leandrolanzieri
as code owners
github-actions
bot
added
Area: doc
crasbe
added
Type: new feature
|
Am I wrong thinking that Clang specific command line options are passed to GCC toolchain ? After a bit of research, Is this a build system behavior triggered by our |
|
@crasbe Hi, could you please tell me if my last assumption is correct and what should I do to fix the build ? |
|
I'm not really sure at which point the As a quickfix, you could add |
|
@crasbe Thanks for your quick reply, much appreciated. I followed your advice; we'll see how it goes. |
|
That looks better now. Some boards will not have enough memory to run the test code, which is what the You can create it with I just ran it on our server and a lot of builds are failing with errors like this: |
|
@crasbe Thanks for the report. It also allowed to track the link time bug you reported. At first glance, it could be that the xipfs library would be compiled with different floating point options ( I have not yet found the culprit, but xipfs library makefile seems to be a good candidate for deeper investigations. More soon. |
|
@crasbe I fixed the build and ran For the record, xipfs built its library in |
|
And I guess you'll have to add a The run on Also it ran without errors, so that's good :) |
|
@crasbe Thank you ! I've run the generation script and added the Makefile.ci to the xipfs example. |
crasbe
left a comment
crasbe
left a comment
There was a problem hiding this comment.
I tried it out on the DWM1001 and found a bug too. When you execute the tests/pkg/xipfs test first and then flash and run the examples/advanced/xipfs application, there is no file listed in the file system. After you execute exec, the file is known to the file system 🤔
2025-07-10 11:25:02,839 # Connect to serial port /dev/ttyACM0
Welcome to pyterm!
Type '/exit' to exit.
vfs df
2025-07-10 11:25:05,464 # vfs df
2025-07-10 11:25:05,469 # Mountpoint Total Used Available Use%
2025-07-10 11:25:05,475 # /dev/nvme0p0 40 KiB 0 B 40 KiB 0%
2025-07-10 11:25:05,481 # /dev/nvme0p1 60 KiB 0 B 60 KiB 0%
> exec
2025-07-10 11:25:23,349 # exec
2025-07-10 11:25:23,517 # Hello World!
> vfs df
2025-07-10 11:25:49,734 # vfs df
2025-07-10 11:25:49,740 # Mountpoint Total Used Available Use%
2025-07-10 11:25:49,746 # /dev/nvme0p0 40 KiB 4 KiB 36 KiB 10%
2025-07-10 11:25:49,752 # /dev/nvme0p1 60 KiB 0 B 60 KiB 0%
Attached you find some review comments. I did not really have time to go through everything as thoroughly as I'd like to.
|
@crasbe Thanks for your review. The behavior you depict is not that a bug, but more a limitation of the implementation. TL; DR : To sum it up, for now, XIPFS is not firmware upgrade resilient. XIPFS relies on flashpage API. So when RIOTS OS gets flashed on the board, this section gets flashed too; and then, former mountpoints are lost. You can trace the mountpoints reservation in In Note : if XIPFS mountpoints data are not persistent on boards flashing, they are regarding to power on/off cycle. |
|
You are right, it even describes the behavior in the function documentation. I assumed that the file was automatically part of the file system after compilation, but of course, it has to be created first. |
|
Is there someone for a code review please ? |
|
@leandrolanzieri @aabadie @MichelRottleuthner @jia200x |
|
I already ping'ed the maintainers in our Matrix chat. Personally I have to allocate some time to give this an actual, thorough review and I did not have the time yet unfortunately :( |
|
@crasbe thank you ! |
|
Please squash :) |
|
@crasbe Thanks for the review and the approval ! I'm facing a Git issue right now. From what I understand, the autosquash were not properly applied. Would you have any tips to perform squashing ? |
|
This is a little bit more complicated unfortunately, because you have a lot of merge commits and you did the What I did to get around this:
git checkout master
git checkout -b xipfs_pr_tmp
This is what the output looks like:
|
|
For the future I would recommend you to avoid adding Merge commits by not using the Github Sync button for the PR branch, but only for the Then you can manually rebase the PR branch on the console with: However in general you don't have to sync the branches at all if no merge conflict arises due to other PRs that were merged. |
GGuche-2XS-Univ-Lille
force-pushed
the
xipfs_pr
branch
from
8c65820 to
8dc5007
Compare
|
@crasbe Thank you a bunch for taking time and coming with this solution. I owe you one. I've followed your procedure and I'll be sticking to your workflow in the future. You saved my day 👍 |
4 participants
Contribution description
This PR aims at integrating the eXecutable In-Place FileSystem into RIOT operating system.
XIPFS implements regular filesystem storage manipulation capabilities, but also allows
to run code almost directly from the FLASH memory.
Just a little bit of RAM is used to relocate what's needed for the execution (data and bss).
It is provided like any other vfs module, and can be called through the defined vfs API.
Nonetheless, it can also be called directly through its own API (xipfs_* functions).
No matter the actual file size is, XIPFS deals with boards' Flash pages granularity.
For example, on Qorvo DWM1001, that means that with two files with different sizes less than 4 Kib,
vfs df will display 8 Kib used.
Due to XIPFS design, directories and files are stored in a similar way.
Echoing the remark above, that also means that an empty directory will cost one flashpage of
Flash memory.
This implementation has been developed and tested on a Decawave DW1001 board (Qorvo).
However, the Flash memory handling relies on available periph/flashpage.h, and should thus
run on any platform supporting addressable Flash memory.
A special constructor has been introduced to make XIPFS compatible with AUX slot mechanism.
As showcased in
tests/pkg/xipfs, usingxipfs_construct_from_flashpagewithmtd_flash_aux_slotwill construct a vfs XIPFS mount point within RIOT's AUX slot memory area.
To build your own fae files, please see XIPFS Format.
Testing procedure
A manual advanced example and automated tests are provided.
Both are great resources to showcase XIPFS api and abilities.
To run the automated tests, please open a terminal with a pyterm, listening to the device.
Open another one and move to the
tests/pkg/xipfsdirectory.Now, please enter the following in this shell :
The first terminal should display something along the lines of following :
Tests are performed twice :
- First pass using a mount point created with respect to
FLASH_WRITABLE_INIT, landing in section.flash_writable.,- Second pass using a mount point created with respect to
mtd_flash_aux_slot, at the end of Flash memory.This can take a little while, as one pass consists of 130+ tests.
To run the manual example, please open a terminal with a pyterm, listening to the device.
Open another one and move to the
examples/advanced/xipfsdirectory.Now, please enter the following in this shell :
Come back to the pyterm terminal.
The resulting display should be similar to :
when XIPFS has already be run and has initialized the two mount points of the filesystem.
Now, please enter help
execis a more a demonstration than a regular shell command.However it will dump an executable file named
hello-world.fae, when none is stored onto/dev/nvme0p0;and run
hello-world.faethen.The embedded file has been compiled for Qorvo dwm1001 board.
Please, use XIPFS Format to compile and build you own FAE file.
Display :
executeis the XIPFS execution shell command to run an executable file.create_executableis the shell command to create an executable_file.Though create_executable takes a name and a bytesize, please remember that it will contain no code until filled by a write command.
This is continued from #21197.