In a previous post I gave the context for my pet project ieee1275-rs, it is a framework to build bootable ELF payloads on Open Firmware (IEEE 1275). OF is a standard developed by Sun for SPARC and aimed to provide a standardized firmware interface that was rich and nice to work with, it was later adopted by IBM, Apple for POWER and even the OLPC XO.

The crate is intended to provide a similar set of facilities as uefi-rs, that is, an abstraction over the entry point and the interfaces. I started the ieee1275-rs crate specifically for IBM’s POWER platforms, although if people want to provide support for SPARC, G3/4/5s and the OLPC XO I would welcome contributions.

There are several ways the firmware takes a payload to boot, in Fedora we use a PReP partition type, which is a ~4MB partition labeld with the 41h type in MBR or 9E1A2D38-C612-4316-AA26-8B49521E5A8B as the GUID in the GPT table. The ELF is written as raw data in the partition.

Another alternative is a so called CHRP script in “ppc/bootinfo.txt”, this script can load an ELF located in the same filesystem, this is what the bootable CD/DVD installer uses. I have yet to test whether this is something that can be used across Open Firmware implementations.

To avoid compatibility issues, the ELF payload has to be compiled as a 32bit big-endian binary as the firmware interface would often assume that endianness and address size.

The entry point

As I entered this problem I had some experience writing UEFI binaries, the entry point in UEFI looks like this:

#![no_main]
#![no_std]
use uefi::prelude::*;

#[entry]
fn main(_image_handle: Handle, mut system_table: SystemTable<Boot>) -> Status {
  uefi::helpers::init(&mut system_table).unwrap();
  system_table.boot_services().stall(10_000_000);
  Status::SUCCESS
}

Basically you get a pointer to a table of functions, and that’s how you ask the firmware to perform system functions for you. I thought that maybe Open Firmware did something similar, so I had a look at how GRUB does this and it used a ppc assembler snippet that jumps to grub_ieee1275_entry_fn(), yaboot does a similar thing. I was already grumbling of having to look into how to embed an asm binary to my Rust project. But turns out this snippet conforms to the PPC function calling convention, and since those snippets mostly take care of zeroing the BSS segment but turns out the ELF Rust outputs does not generate one (although I am not sure this means there isn’t a runtime one, I need to investigate this further), I decided to just create a small ppc32be ELF binary with the start function into the top of the .text section at address 0x10000.

I have created a repository with the most basic setup that you can run. With some cargo configuration to get the right linking options, and a script to create the disk image with the ELF payload on the PReP partition and run qemu, we can get this source code being run by Open Firmware:

#![no_std]
#![no_main]

use core::{panic::PanicInfo, ffi::c_void};

#[panic_handler]
fn _handler (_info: &PanicInfo) -> ! {
    loop {}
}

#[no_mangle]
#[link_section = ".text"]
extern "C" fn _start(_r3: usize, _r4: usize, _entry: extern "C" fn(*mut c_void) -> usize) -> isize {
    loop {}
}

Provided we have already created the disk image (check the run_qemu.sh script for more details), we can run our code by executing the following commands:

$ cargo +nightly build --release --target powerpc-unknown-linux-gnu
$ dd if=target/powerpc-unknown-linux-gnu/release/openfirmware-basic-entry of=disk.img bs=512 seek=2048 conv=notrunc
$ qemu-system-ppc64 -M pseries -m 512 --drive file=disk.img
[...]
  Welcome to Open Firmware

  Copyright (c) 2004, 2017 IBM Corporation All rights reserved.
  This program and the accompanying materials are made available
  under the terms of the BSD License available at
  http://www.opensource.org/licenses/bsd-license.php


Trying to load:  from: /vdevice/v-scsi@71000003/disk@8000000000000000 ...   Successfully loaded

Ta da! The wonders of getting your firmware to run an infinite loop. Here’s where the fun begins.

Doing something actually useful

Now, to complete the hello world, we need to do something useful. Remeber our _entry argument in the _start() function? That’s our gateway to the firmware functionality. Let’s look at how the IEEE1275 spec tells us how we can work with it.

This function is a universal entry point that takes a structure as an argument that tells the firmware what to run, depending on the function it expects some extra arguments attached. Let’s look at how we can at least print “Hello World!” on the firmware console.

The basic structure looks like this:

#[repr(C)]
pub struct Args {
  pub service: *const u8, // null terminated ascii string representing the name of the service call
  pub nargs: usize,       // number of arguments
  pub nret: usize,        // number of return values
}

This is just the header of every possible call, nargs and nret determine the size of the memory of the entire argument payload. Let’s look at an an example to just exit the program:

#[no_mangle]
#[link_section = ".text"]
extern "C" fn _start(_r3: usize, _r4: usize, entry: extern "C" fn(*mut Args) -> usize) -> isize {
    let mut args = Args {
        service: "exit\0".as_ptr(),
        nargs: 0,
        nret: 0
    };

    entry (&mut args as *mut Args);
    0 // The program will exit in the line before, we return 0 to satisfy the compiler
}

When we run it in qemu we get the following output:

Trying to load:  from: /vdevice/v-scsi@71000003/disk@8000000000000000 ...   Successfully loaded
W3411: Client application returned.

Aha! We successfully called firmware code!

To be continued…

To summarize, we’ve learned that we don’t really need assembly code to produce an entry point to our OF bootloader (tho we need to zero our bss segment if we have one), we’ve learned how to build a valid OF ELF for the PPC architecture and how to call a basic firmware service.

In a follow up post I intend to show a hello world text output and how the ieee1275 crate helps to abstract away most of the grunt to access common firmware services. Stay tuned!

I have been doing random coding experiments with my spare time that I never got to publicize much outside of my inner circles. I thought I would undust my blog a bit to talk about what I did in case it is useful for others.

For some background, I used to manage the bootloader team at Red Hat a few years ago alongside Peter Jones and Javier Martinez. I learned a great deal from them and I fell in love with this particular problem space and I have come to enjoy tinkering with experiments in this space.

There many open challenges in this space that we could use to have a more robust bootpath across Linux distros, from boot attestation for initramfs and cmdline, A/B rollbacks, TPM LUKS decryption (ala BitLocker)…

One that particularly interests me is unifying the firmware-kernel boot interface across implementations in the hypothetical absence of GRUB.

Context: the issue with GRUB

The priority of the team was to support RHEL boot path on all the architectures we supported. Namely x86_64 (legacy BIOS & UEFI), aarch64 (UEFI), s390x and ppc64le (Open Power and PowerVM).

These are extremely heterogeneous firmware interfaces, some are on their way to extinction (legacy PC BIOS) and some will remain weird for a while.

GRUB, (GRand Unified Bootloader) as it names stands, intends to be a unified bootloader for all platforms. GRUB has to support a supersetq of firmware interfaces, some of those, like legacy BIOS do not support much other than some rudimentary support disk or network access and basic graphics handling.

To get to load a kernel and its initramfs, this means that GRUB has to implement basic drivers for storage, networking, TCP/IP, filesystems, volume management… every time there is a new device storage technology, we need to implement a driver twice, once in the kernel and once in GRUB itself. GRUB is, for all intent and purposes, an entire operating system that has to be maintained.

The maintenance burden is actually quite big, and recently it has been a target for the InfoSec community after the Boot Hole vulnerability. GRUB is implemented in C and it is an extremely complex code base and not as well staffed as it should. It implements its own scripting language (parser et al) and it is clear there are quite a few CVEs lurking in there.

So, we are basically maintaining code we already have to write, test and maintain in the Linux kernel in a different OS whose whole purposes (in the context of RHEL, CentOS and Fedora) its main job is to boot a Linux kernel.

This realization led to the initiative that these days are taking shape in the discussions around nmbl (no more boot loader). You can read more about that in that blog post, I am not actively participating in that effort but I encourage you to read about it. I do want to focus on something else and very specific, which is what you do before you load the nmble kernel.

Booting from disk

I want to focus on the code that goes from the firmware interface to loading the kernel (nmbl or otherwise) from disk. We want some sort of A/B boot protocol that is somewhat normalized across the platforms we support, we need to pick the kernel from the disk.

The systemd community has led some of the boot modernization initiatives, vocally supporting the adoption of UKI and signed pre-built initarmfs images, developing the Boot Loader Spec, and other efforts.

At some point I heard Lennart making the point that we should standardize on using the EFI System Partition as /boot to place the kernel as most firmware implementations know how to talk to a FAT partition.

This proposal caught my attention and I have been pondering if we could have a relatively small codebase written in a safe language (you know which) that could support a well define protocol for A/B booting a kernel in Legacy BIOS, S390 and OpenFirmware (UEFI and Open Power already support BLS snippets so we are covered there).

My modest inroad into testing this hypothesis so far has been the development of ieee1275-rs, a Rust module to write programs for the Open Firmware interface, so far I have not been able to load a kernel by myself but I still think the lessons learned and some of the code could be useful to others. Please note this is a personal experiment and nothing Red Hat is officially working on.

I will be writing more about the technical details of this crate in a follow up blog post where I get into some of the details of writing Rust code for a firmware interface, this post is long enough already. Stay tuned.

It has been a while since my last post, so I figured I just picked up a topic that has been around my mind lately.

After I ported the RSVG Pixbuf Loader to Rust (although I gave up the meson-fu to Federico and Bilal) I decided that maybe I should give a try at porting the WebP Pixbuf Loader.

webp-pixbuf-loader is probably the only FOSS project I have started on my own that I have managed to maintain in the long run without orphaning or giving it up to someone else. I wrote it out of curiosity and it turns out plenty of people find it rather useful as webp is pretty popular for storing comic book backups and other media.

The WebP Pixbuf Loader is relatively small although ever since animation support was contributed it grew quite a bit. I’ve been handling a couple of issues ranging from endianness to memory leaks, I thought it was probably worth the while to give it some Rusty love.

Porting the static image support was relatively quick, there’s, but it took me a while to understand how animation works in GDK-Pixbuf as the original animation support in C was contributed by alanhaw.

I suspect I am prolly the first person to use the GdkPixbufLoader APIs to implement a new pixbuf loader, I had request a few fixes upstream, kudos to Sebastian Dröge and Bilal for handling those swiftly and shipping them in the last release.

Anyhow, last month I finally made it all work:

Hesitations

Now comes the hesitation part, regardless of integrating the former tests in the build system (Cargo is great at embedded unit testing but meson is better at running an integration test plan), my main gripe is that it turns out that there’s quite a few people packaging this, not just for Linux distros but also BSDs, Illumos, Windows and brew/macOS…

I really don’t know what the impact would be for anyone packaging outside of the Linux world, I have a few CI pipelines for Windows but obviously I am about to break them if I switch.

I am pondering the idea of releasing a bunch of beta releases and hoping package maintainers will start taking notice that I’m moving on, but I am trying to be mindful of how much time they need to sink for the Rust move to happen and weight that against the net benefit.

The other part that makes me hesitate over flipping the switch is the measure of the overall benefit. Sure Rust is nicer to maintain, but it still is a small codebase, Rust adds a bunch of runtime dependencies (bindings) and it is not clear to me how healthy the webp bindings are going to be long term, there are two similarly named bindings one has more frequent releases and the other is more complete which is annoying. These bindings bring an issue of size: the Rust port is 4MB in size vs. 76KB for the C implementation.

Not sure what to do, feel free to add your thoughts in the comment section.

UPDATE: It looks like the size issue can be addressed and total size drops down to ~300KB by using the right flags.

I’ve been toying around with Rust during Easter. It has been a while since I last had a go at it for UEFI binaries. Turns out that the uefi-rs crate has gotten tons of love in terms of usability, stability and built-in protocol interfaces.

Now, no_std is useful for a myriad of use cases like embedded platforms, it can even work in environments with no memory allocation. You can find an example UEFI app here, it is nothing fancy, crawls the directory tree of the main EFI System Partition.

For the purpose of my pet project I wanted to add a Boot Loader Spec parser to my project that I could also link in a library with std:: as well as no_std. This introduces the requirement that your no_std environment needs to be hooked up with an allocator, at which point you can consume the alloc crate.

Otherwise you can only use the data types in libcore (as well as crates that only depend on libcore), which is a small subset of std stuff. uefi-rs sets this up for you but this documentation might come handy.

This would be your minimal no_std lib.rs:

#[no_std]
fn my_function () -> bool {
    true
}

to std or no to std

Now, the first problem we have here is that we want to also be able to compile this module with the standard library. How can we do both? Well, turns out there is a convention for this, enter cargo features:

[features]
default = ["std"]
std = []

At which point you can use #![cfg_attr] to make no_std a conditional attribute upon compilation in the absence of the std feature.

#![cfg_attr(not(feature = "std"), no_std)]
fn my_function () -> bool {
    true
}

And you can drop the std feature at compile time like this

$ cargo build --no-default-features

Or consume the crate as a dependency in your toml like this:

[dependencies]
nostdable-crate = { version = "0.1.1", default-features = false }

optionally use core:: and alloc::

So, let’s say we have the following function:

#![cfg_attr(not(feature = "std"), no_std)]

fn my_function () -> String {
    String::from("foobar")
}

Here we are using std::String which is not available, this is going to fail on a no_std setup as is. libcore’s core::str will only work for static strings as there is no allocation available. In these environments you need an allocator so that we can import the alloc:: crate, if you have an allocator primitive you need to implement a GlobalAlloc and initialize it with #[global_allocator]. This is no needed in UEFI so I didn’t have to do it.
So the question is, how do I set things up so that I can use core and alloc types conditionally? This would be it:

#![cfg_attr(not(feature = "std"), no_std)]

// SETUP ALLOCATOR WITH #[global_allocator]
// see: https://doc.rust-lang.org/std/alloc/trait.GlobalAlloc.html

#[cfg(not(feature = "std"))]
extern crate alloc;

#[cfg(not(feature = "std"))]
use alloc::string::String;

fn my_function () -> String {
    String::from("foobar")
}

If you are using Vec<> the same applies, you’d have to conditionally use it from alloc:: or from std:: accordingly.

Conclusions

I really thought that porting an existing crate to #[no_std] was a lot more work and a lot more constraining. In general, if you depend or could port your code to stuff that is present in both std:: and core:: + alloc:: you should be good to go. If you want to target an environment where an allocator is not possible then porting relatively common Rust code becomes a lot more complicated as you need to find a way to write your code with no allocations whatsoever.

In my original implementation I did some file io:: operations so my entire crate API was filled with -> io::Result<,>. io:: extirpation was 90% of the porting efforts as I didn’t have any external dependencies. If you have a crate that relies on a complicated dependency tree you might have a harder time porting your existing code.

If you want to have a look at my Boot Loader Spec crate for a full example it’s in my gitlab.

Changes are risky, taking on a new role on a new company with people you never worked before, growing a whole org from scratch is hard work that comes with a lot of uncertainties. When I decided that I wanted to try something new and join Amazon to work on the new Kindle site in Madrid I knew that it was a leap of faith. I’ve met amazing people and I’ve learned a lot about why Amazon is so successful as a consumer focused company, this is the first time I’ve joined a company to work on closed source software full time and that change has taken a bigger toll that I anticipated, so for a while I’ve been looking for a change. Dealing with this on top of raising a 2 year old while moving cities plus the COVID19 lockdown hasn’t made things any easier for me and my family either.

Luckily I didn’t have to look much further, when I mentioned to Nacho Casal from gedit/GtkSourceView fame that I was looking into something different he mentioned that the NICE DCV team within AWS HPC org was looking for an engineering manager. Suffice to say, I did the interviews, they went well and since mid August I’ve been part of this amazing team. I am peer with Paolo Borelli and I report to Paolo Maggi both former GNOME/gedit/GtkSourceView maintainers. And to add the cherry on top my skip level manager is Ian Colle from Inktank’s and also an ex-RedHatter. The team has made me feel at home.

DCV is a propietary remote desktop solution optimized for high resolution and low latency usecases, it is an amazing piece of technology and it is the most competitive remote desktop protocol for the Linux desktop. It builds upon many GNOME tecnologies like GTK for our Linux/Windows/macOS clients, GStreamer and recently the team has been making inroads into adopting Rust. Stack wise this is a very exciting job for me as it touchs pretty much all the areas I care about and they do their best to open source stuff when they can.

The scope of my team is going to cover mostly the customer facing deliverables such as the clients, packaging and other release process duties. However I will be coordinating upstream contributions as well which is pretty exciting, I am looking forward to work on Wayland integration and other GTK niceties as priority allows. The team understands the importance on investing in the sustainability of the FOSS projects we rely on and I want to make sure that is the case.

Happy hacking!

Everybody knows that I have always been a firm believer in Gtk+’s potential to be a great cross platform toolkit beyond Linux. GIMP and Inkscape, as an example, are loyal users that ship builds for those platforms. The main challenge is the short amount of maintainers running, testing and improving those platforms.

Gtk+ has a few shortcomings one of them, one of the biggest ones is lack of a11y support outside of Linux. Since I have regular access to a modern OSX machine I decided to give this a go (and maybe learn son Obj-C in the process).

So I started by having a look at how ATK works and how it relates to the GTK DOM, my main goal was to have a GTK3 module that would walk through the toplevels and build an OSX accessibility tree.

So my initial/naive attempt is in this git repo, which you can build by installing gtk from brew.

Some of the shortcomings that I have found to actually test this and move forward:

• Running gtk3-demo creates an NSApp that has no accessibility enabled, you can tell because the a11y inspector that comes with XCode won’t show metadata even for the window decorator controls. I have no idea how to enable that manually, it looks like starting an actual NSApp, like Inkscape and GIMP do would give you part of that.
• Inkscape and GIMP have custom code to register themselves as an acutal NSApp as well as override XDG env vars in runtime to set the right paths. I suspect this is something we could move to G and GtkApplication.
• The .dylib I generate with this repo will not actually load on Inkscape for some reason, so as of now I am stuck with having to somehow build a replacement gtk dylib for Inkscape with my code instead of through an actual module.

So this is my progress thus far, I think once I get to a point where I can iterate over the concept, it would be easier to start sketching the mapping between ATK and NSAccessibility. I would love feedback or help, so if you are interested please reach out by filing an issue on the gitlab project!

Since the beginning of December I started working for Amazon in Madrid to run the team responsible for the Kindle experience on PC and the web. This is a bit of a career shift for me, for a while I’ve been wondering how would it be like to deliver a user experience of a successful consumer product, I have always been working on the seat of the OS/platform provider and more recently I have learned the hardware end by working with OEMs as part of my last role at Red Hat. However I have never been in the shoes of an ISV delivering an app for a vertical market, on top of that the position was advertised for Madrid for an onsite team, while working from home has many advantages running an entire team onsite is also a refreshing change for me, all in all it seemed like a cool opportunity where I could learn lots and try something different so I made the jump.

By the way, my team is hiring software engineers in Madrid, Spain, so if this is an area you are interested in please DM me on twitter.

This is an extract of an email I just sent internally at Red Hat that I wanted to share with the wider GNOME and Freedesktop community.

After 6+ wonderful years at Red Hat, I’ve decided to hang the fedora to go and try new things. For a while I’ve been craving for a new challenge and I’ve felt the urge to try other things outside of the scope of Red Hat so with great hesitation I’ve finally made the jump.

I am extremely proud of the work done by the teams I have had the honour to run as engineering manager, I met wonderful people, I’ve worked with extremely talented engineers and learned lots. I am particularly proud of the achievements of my latest team from increasing the bootloader team and improving our relationship with GRUB upstream, to our wins at teaching Lenovo how to do upstream hardware support to improvements in Thunderbolt, Miracast, Fedora/RHEL VirtualBox guest compatibility… the list goes on and credit goes mostly to my amazing team.

Thanks to this job I have been able to reach out to other upstreams beyond GNOME, like Fedora, LibreOffice, the Linux Kernel, Rust, GRUB… it has been an amazing ride and I’ve met wonderful people in each one of them.

I would also like to make a special mention to my manager, Christian Schaller, who has supported me all the way in several ways both professionally and personally. There is this thing people say: “people do not leave companies, they leave managers”. Well this is certainly not the case, in Christian I have found not only a great manager but a true friend.

As for my experience at Red Hat, I have never lasted more than 2 years in the same spot before, I truly found my place there, deep in my heart I know I will always be a Red Hatter, but there are some things I want to try and learn elsewhere. This job switch has been the hardest departure I ever had and in many ways it breaks my heart to leave. If you are considering joining Red Hat, do not hesitate, there is no better place to write and advocate for Free Software.

I will announce what I will be doing next once I start in December.

I have recently created a C API for a library dealing with Boot Loader Spec files as well as the GRUB environment file. In the process I have learnt a few things that, coming from a C background, were not obvious to me at all.

Box to control unsafe ownership

Say we have this simple Rust API:

pub struct A {
  counter: u8
}

impl A {
  pub fn new(count: u8) -> A {
    A { counter: count }
  }
}

Let’s start with the new method wrapper:

#[no_mangle]
pub extern "C" fn a_new(count: u8) -> *mut A {
  let boxed_a = Box::new(A {counter: count});
  Box::into_raw(boxed_a)
}

A Box is basically a smart pointer, it allows us to control the lifetime of the data outside of the boundaries of Rust’s borrow checker. Box::into_raw returns a pointer to the allocated A instance. Let’s see how to access that data again:

#[no_mangle]
pub extern "C" fn a_get_counter(a: *mut A) -> u8 {
  let a = unsafe { Box::from_raw(a) };
  let count = a.counter;
  Box::into_raw(a);
  count
}

Box::from_raw is an unsafe method that turns a pointer into an owned Box, this allows us to access the pointer data safely from Rust. Note that Box is automatically dereferenced.

UPDATE: Sebastian Dröge has rightly pointed out that the above method is wrong, note that this is how I found most StackOverflow and other people explain how I should use the data again, but if Sebastian says it is wrong then I know it to be wrong ;-).

Turns out that casting the pointer as a reference inside an unsafe block is enough:

#[no_mangle]
pub unsafe extern "C" fn a_get_counter(a: *mut A) -> u8 {
  let a = &*a;
  a.counter
}

Now we need to give the C user a deallocator for instances of A, this is relatively straightforward, we wrap the object around a Box and since we don’t call into_raw again, as soon as the Box is out of scope the inner contents are dropped too:

#[no_mangle]
pub unsafe extern "C" fn a_drop(a: *mut A) {
  Box::from_raw(a);
}

Strings

In Rust there are two standard ways to interact with strings, the String type, a dynamic utf-8 string that can be modified and resized, and &str, which basically is a bare pointer to an existing String. It took me a while to realize that internally a String is not null terminated and can contain many null characters. This means that the internal represenation of String is not compatible with C strings.

To address this, Rust provides another two types and referenced counterparts:

• OsString and &OsStr: a native string tied to the runtime platform encoding and sizing
• CString and &CStr: a null terminated string

My main grudge with this model when creating a C API is that it creates friction with the C boundary for a couple of reasons:

• Internal Rust APIs often expect String or &str, meaning that at the C API boundary you need to allocate a CString if you use String as the internal representation, or the other way around if you use CString as the internal representation
• You can’t “transparently” pass ownership of a CString to C without a exposing a deallocator specific to CString, more on this on the next section.

This means that compared to a C implementation of the API your code will liekly use more allocations which might or might not be critical depending on the use case, but this is something that struck me as a drawback for Rustification.

UPDATE: I am talking solely about the C API boundary, Rust is _great_ at giving you tools to avoid extra allocations (yay slices!), you can create a parser of a large chunk of text without allocating any extra strings to chunk the source text around.

Allocator mismatch

Something else I stumbled upon was that Rust does not use malloc/free, and that mismatch has annoying side effects when you are trying to rustify an API. Say you have this C code:

char* get_name() {
  const char* STATIC_NAME = "John";
  char* name = (char*)malloc(sizeof(STATIC_NAME));
  memcpy(name, STATIC_NAME, sizeof(STATIC_NAME));
}

int main () {
  char * name = get_name();
  printf("%s\n", name);
  free(name);
  return 0;
}

Now if you want to Rustify that C function, the naive way (taking into account the String vs. CString stuff I mentioned before) would be to do this:

#[no_mangle]
pub extern "C" fn get_name() -> *mut std::os::raw::c_char {
  const STATIC_NAME: &str = "John";
  let name = std::ffi::CString::new(STATIC_NAME)
               .expect("Multiple null characters in string");
  name.into_raw()
}

But this is not exactly the same as before, note that in the C example we call free() in order to drop the memory. In this case we would have to create a new method that calls CString::from_raw() but that won’t be compatible with the original C API.

This is the best I was able to came up with:

/* You can use the libc crate as well */
extern {
  fn malloc(size: usize) -> *mut u8;
  fn memcpy(dest: *mut u8, src: *const u8, size: usize) -> *mut u8;
}

#[no_mangle]
pub extern "C" fn get_name() -> *mut u8 {
  const STATIC_NAME: &str = "John";
  let name = std::ffi::CString::new(STATIC_NAME)
               .expect("Multiple null characters in string");
  let length = name.as_bytes_with_nul().len();
  let cname = unsafe { malloc(length) };
  unsafe { memcpy(cname, name.as_bytes_with_nul().as_ptr(), length) };
  cname
}

Note that STATIC_NAME is just an example, usually the data comes from a String/&str in your Rust API. The problem here is that we allocated an extra CString to then copy its contents using malloc/memcpy and then drop it immediately.

However, later, while working on creating UEFI binaries from Rust, I learned that Rust allows you to override its own allocator and use a custom one or the native system one. This would be another way to achieve the same and save the malloc/memcpy step, but don’t trust me 100% here as I am not sure whether this is entirely safe (if you know, let me know in the comments).

UPDATE: Many people have pointed out that overriding the allocator to use the system is absolutely fine:

use std::alloc::System;

#[global_allocator]
static GLOBAL: System = System;

#[no_mangle]
pub extern "C" fn get_name() -> *mut u8 {
  const STATIC_NAME: &str = "John";
  let name = std::ffi::CString::new(STATIC_NAME).expect("Multiple null characters in string");
  name.into_raw() as *mut u8
}

Traits as fat pointers

Let’s say we have the following API with two types and a trait implemented by both:

pub struct A {}
pub struct B {}

impl A {
  pub fn new () -> A { A{} }
}

impl B {
  pub fn new () -> B { B{} }
}

pub trait T {
  fn get_name(&self) -> std::ffi::CString;
}

impl T for A {
  fn get_name(&self) -> std::ffi::CString {
    std::ffi::CString::new("I am A").expect("CString error")
  }
}

impl T for B {
  fn get_name(&self) -> std::ffi::CString {
    std::ffi::CString::new("I am B").expect("CString error")
  }
}

Now the problem is, if we want a single wrapper for T::get_name() to avoid having to wrap each trait implementation family of functions, what do we do? I banged my head on this trying to Box a reference to a trait and other things until I read about this in more detail. Basically, the internal representation of a trait is a fat pointer (or rather, a struct of two pointers, one to the data and another to the trait vtable).

So we can transmute a reference to a trait as a C struct of two pointers, the end result for type A would be like this (for B you just need another constructor and cast function):

#[repr(C)]
pub struct CTrait {
  data: *mut std::os::raw::c_void,
  vtable: *mut std::os::raw::c_void
}

#[no_mangle]
pub extern "C" fn a_new() -> *mut A {
  Box::into_raw(Box::new(A::new()))
}

#[no_mangle]
pub extern "C" fn a_drop(a: *mut A) {
  unsafe{ Box::from_raw(a) };
}

#[no_mangle]
pub extern "C" fn a_as_t (a: *mut A) -> CTrait {
  let mut boxed_a = unsafe { Box::from_raw(a) };
  let ret: CTrait = {
    let t: &mut dyn T = &mut *boxed_a;
    unsafe { std::mem::transmute::<&mut dyn T,CTrait> (t) }
  };
  Box::into_raw(boxed_a);
  ret
}

#[no_mangle]
pub extern "C" fn t_get_name(t: CTrait) -> *mut u8 {
  let t = unsafe { std::mem::transmute::<CTrait, &mut dyn T> (t) };
  t.get_name().into_raw() as *mut u8
}

The C code to consume this API would look like this:

typedef struct {} A;
typedef struct {
  void* _d;
  void* _v;
} CTrait;

A*      a_new();
void    a_drop(A* a);
CTrait  a_as_t(A* a);
char*   t_get_name(CTrait);

int main () {
  A* a = a_new();
  CTrait t = a_as_t(a);
  char* name = t_get_name(t);
  printf("%s\n", name);
  free(name);
  a_drop(a);
  return 0;
}

Error reporting

Another hurdle has been dealing with Result<> in general, however this is more of a shortcoming of C’s lack of standard error reporting mechanism. In general I tend to return NULL to C API calls that expect a pointer and let C handle it, but of course data is lost in the way as the C end has no way to know what exactly went wrong as there is no error type to query. I am tempted to mimick GLib’s error handling. I think that if I was trying to replace an existing C library with its own error reporting mapping things would become easier.

Conclusions

I am in love with Rust and its ability to impersonate C is very powerful, however it is note entirely 0 cost, for me, the mismatch between string formats is the biggest hurdle as it imposes extra allocations, something that could become really expensive when rustifying C code that passes strings back and forth from/to the API caller. The other things I mentioned were things that took me quite some time to realize and by writing it here I hope I help other people that are writing Rust code to expose it as a C API. Any feedback on my examples is welcome.

We’re about to finish the three days long first GNOME Performance Hackfest here in Cambridge.

We started covering a few topics, there are three major areas we’ve covered and in each one of those there has been a bunch of initiatives.

GNOME Shell performance

Jonas Adahl, Marco Trevisan, Eric Anholt, Emmanuele Bassi, Carlos Garnacho and Daniel Stone have been flocking together around Shell performance. There has been some high level discussions about the pipeline, Clutter, Cogl, cairo/gtk3 and gtk4.

The main effort has been around creating probes across the stack to help Christian Hergert with sysprof (in drm, mutter, gjs…) so that we can actually measure performance bottlenecks at different levels and pinpoint culprits.

We’ve been also looking at the story behind search providers and see if we can rearchitect things a bit to have less roundtrips and avoid persistent session daemons to achieve the same results. Discussions are still ongoing on that front.

GNOME Session resource consumption

Hans de Goede put together a summary of the resource consumed in a typical GNOME session in Fedora and tweaks to avoid those, you can check the list in the agenda.

There are some issues specific to Fedora there, but the biggest improvement that we can achieve is shutting down’s GDM’s own gnome-shell instance, for which Hans already has a working patch. This should reduce resource consumption by 280megs of RAM.

The second biggest target is GNOME Software, which we keep running primarily for the shell provider. Richard Hughes was here yesterday and is already working on a solution for this.

We are also looking into the different GNOME Settings Daemon processes and trying to figure out which ones we can shut down until needed.

Surely there’s stuff I’ve missed, and hopefully we’ll see blogposts and patches surfacing soon after we wrap up the event. Hopefully we can follow up during GUADEC and start showing the results.

On Tuesday we enjoyed some drinks out kindly hosted by Collabora.

I’d like to thank the Eben Upton and the Raspberry Pi Foundation for sponsoring the venue and sending Eric Anholt over.