Any module which is sold by itself and not with a developer’s kit carrier board is called a “commercial” module. Such modules have an improved warranty, and require an added carrier board. All of the small form factor development kit modules do not have eMMC, and do have SD card slots. Having eMMC instead of SD card slot on the module itself means it is a commercial module, and it also means flashing and firmware are different than a dev kit. This is the reason why I’m trying to to nail this down as commercial or developer module. If you are looking at software and instructions, then consider this a commercial module.
I mentioned earlier that Jetsons do not have a hardware BIOS. If this were developer’s kit, then this would be in the QSPI memory of the module. As a commercial Orin Nano with eMMC, then that content is instead flashed into signed partitions of the eMMC. For eMMC models the default signature is a NULL key, but they are still signed, and if not signed, those partitions are rejected. Some of the content can be retrieved from “/boot”, e.g., the kernel Image file and device tree for cases where they are named in the “/boot/extlinux/extlinux.conf” file. If security fuses are burned (which SD card model of Orin Nano does not have), then much of the “/boot” content is ignored and is only allowed from a signed partition.
I mention this because booting can use either the “/boot” content or the binary content in the signed partitions. If the “/boot” content is named in extlinux.conf, but not found, or simply not named, then the content comes from the partition content. When you flash the Image and device tree are added to partitions even if present in “/boot”. Update via OTA I think understands this mechanism. The A/B backup rootfs partition scheme should understand this as well (I’ve not actually worked on that so I don’t know details, someone from NVIDIA would have to comment). In essence, for a commercial module, OTA must sign any of the partition content and then update those non-rootfs partitions (using the correct key; NULL for default case). In the case of an A/B backup rootfs partition scheme, then it also has to update that content (if any boot content changes) on both partitions. I’m not sure how A/B switchover works in combination with a PXE boot, but it is likely you are not using the A/B scheme.
Are you merely using PXE boot, or are you also using an A/B rootfs partition backup scheme? Can we confirm there is no other external boot device involved, including USB or NVMe boot devices once you no longer use the PXE? My assumption is that you do not normally run on PXE, and that this is only intended to update the module for non-PXE boot while skipping having the Jetson in recovery mode.
What I suspect you need to do is to completely set up your PXE boot, and to have partitions in the eMMC. The non-rootfs content would always be in the eMMC for the Orin Nano commercial module (QSPI for the dev kit module which does not have eMMC). I’m going to explain a bit about the switch_root and pivot_root commands in order to explain what might or might not work for update over PXE such that it updates the Jetson instead of the master node which PXE would refer to. If this works you are going to have to do a lot of testing and experimentation and it is by no means guaranteed, but the idea is similar to how an initrd image works (so I’ll describe more on initrd).
If you have pivot_root and/or switch_root, then you should be able to see their descriptions via:
man pivot_rootman switch_root
In the earliest of days of Linux there were no kernel modules. Any filesystem type support mandated this be integrated directly into the kernel, and every kernel change mandated reboot to test. There was no initial ramdisk, and it wasn’t needed. The only filesystem types supported back then were minix and ext2. The boot chain understood those types natively (there was no GRUB or GRUB2 either). As soon as modules were introduced it became possible to require a filesystem to read the kernel and modules from which itself contained the information needed to read that filesystem…it was the proverbial “Catch 22” dilemma. The initrd was invented to act as an adapter; this super simple filesystem type could always be understood, and it could contain any modules required to read that filesystem type. The boot chain no longer needed to read the filesystem which the kernel itself reads, and so new filesystem types did not need to be added to the boot chain itself.
If you are going to use PXE mounts to perform updates, then you’re going to need to run something similar to how the initrd was used. Booting to the initrd, loading modules needed, and then pivoting the rootfs to the real filesystem is how you will need to work as well. Along with the initrd the pivot_root function was added (and now there is also a switch_root). See if your environment has them:
which switch_rootwhich pivot_root- Or look in “
/usr/sbin/” since your path might not include that, and “ls *_root”.
- If you have man pages for development (“
sudo apt-get install manpages-dev” if you have the repository but not the man pages), then “man -a pivot_root” and “man -a switch_root”. You can do a Google search for for “man page” and “pivot_root”.
For the case of the filesystem being in a RAM disk (the initrd is init on a RAM disk) the init script is run, and when modules are loaded (it doesn’t have to be modules loading, this is where you customize), then once the environment is set up to perform whatever equivalent to flash you want, then you’ll need to have a way to perform the OTA not to the existing PXE mount, but instead to the eMMC partitions (presumably there will be some version of the o/s there already; if not, then your init will do something like rsync that content into the eMMC instead of the PXE content). Then the existing root of the filesystem (directory “/”) will need to pivot or switch from the PXE content to the eMMC content (if you have an A/B scheme this is more complicated because then your eMMC will have both “A” and “B” partitions), and at that point perform any additional updates you might need. The pivot_root and switch_root programs are for this purpose: They swap out the existing disk device behind “/” and can install the eMMC partition to instead be the “/”.
At the point of switching from PXE “/” to eMMC “/” (the rootfs switchover) the init process of the new filesystem begins to run. Thus I’ll provide some details about init which will probably bore you to sleep!
When the kernel itself runs it is the user “root” who owns this, and the process ID is zero. The kernel runs one and only one process, init, and that is PID 1. Everything else which runs is a result of actions taken within init. In the early days init was nothing but a bash script. This is still true within an initrd, it is basically just a bash script which loads kernel modules and then performs a pivot or switch root to go to the real filesystem. The original init script goes away (it ends) and the filesystem on the new rootfs begins, and so this is still PID 1 (kernel is PID 0). Nowadays the init of the real filesystem is systemd instead of being a bash script, and it has services and such…it is a fancier bash script with a different organization. If you run the command “ls -l /usr/sbin/init”, then you will see this is just a symbolic link to systemd.
The reason this becomes important is that you have to create your own custom init script within the init of the PXE boot environment. This has to put content into the eMMC partition (or partitions if A/B scheme), set up the environment (maybe if there is also an OTA script and not just a flash of a previously empty Jetson, in which case setup is different), followed by a pivot_root or switch_root to the actual eMMC partition. If the script adds or edits any content in a non-rootfs partition, then it has to be able to sign with the correct key (and a NULL key is default, but it is still signed).
If you plan to try this, then the place to start is to boot from PXE, and then begin customizing the init script on the master host PC serving the PXE content of the “/” partition. You have to make a simple script (you don’t need systemd, you can instead use a bash script; the bash program itself might or might not need to be simplified to something like busybox, but since you have the space of a real disk on the host PC this shouldn’t be required) to basically install the rootfs from the correct source to the eMMC partition. When done, it needs to pivot or switch to eMMC and continue boot. The part on eMMC is just purely unmodified Jetson content. The part which differs is (A) getting it to boot to PXE to start with, and (B) a modified init in the PXE content to correctly copy content to eMMC.
It’s quite a lot of work to understand, but in reality, the idea is probably simple. You just have to get past all of the understanding required for customizing an initrd which will instead be a PXE filesystem rather than an initial RAM disk.