I'm unable to change the font of the text in the SVG.
I tried to run FC_DEBUG and it looks like no matter what is selects Liberation Serif.

I have fontconfig version 2.12.6 installed on my system.

This is the SVG I'm using
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 108 16" version="1.1"><text font-family="sans-serif" font-size="12px" x="54" y="12" fill="#000000" text-anchor="middle" >Fancy Text</text></svg>

'use strict';
const sharp = require('sharp');
const fs = require('fs');

let buffer = '<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 108 16" version="1.1"><text font-family="sans-serif" font-size="12px" x="54" y="12" fill="#000000" text-anchor="middle" >Fancy Text</text></svg>';

sharp(Buffer.from(buffer))
    .toBuffer().then(buffer => {
        fs.writeFile('Sans-serif.png', buffer, (err) => { });
});

If I view the same SVG in the browser it looks correct.

-- Esben N.