SSR: non-ASCII characters in static template text make the whole rendered page a two-byte string #4328

What happens

A single non-ASCII character in static JSX text, such as the → in <a>View →</a>, makes the whole server-rendered page a two-byte string in V8. V8 stores a string at one byte per character only while every character is Latin-1, so once the → is concatenated in, the flattened page becomes UTF-16. That roughly doubles the cost of flattening the rendered rope and makes the TextEncoder.encode call in the response stream several times slower.

I found this while comparing Solid 2.0 (rc.19, @solidjs/vite-plugin next.51, router next.42) with SolidStart on the e18e framework-tracker benchmark app. Its /server-side-rendered page is a 1000-row table where each row ends in <a href={...}>View →</a>. The rendered page is about 300 KB.

On that page, TextEncoder.encode on the response takes about 1.3 ms of the roughly 3.4 ms of CPU per request: about 0.5 ms for V8 to flatten the rope and about 0.7 ms for the UTF-16 to UTF-8 conversion. The same encode on an ASCII-only version of the page is much cheaper. On a flat 170 KB string, TextEncoder.encode takes 0.06 ms when it is all ASCII and 0.35 ms when it contains one →.

How to reproduce

The effect is visible without Solid:

const te = new TextEncoder()
const page = (arrow) =>
  Array.from({ length: 1000 }, () =>
    `<tr><td>${crypto.randomUUID()}</td><td><a href="/x/${crypto.randomUUID()}">View ${arrow ? '→' : '-&gt;'}</a></td></tr>`
  ).join('')

for (const arrow of [false, true]) {
  const strings = Array.from({ length: 200 }, () => page(arrow))
  const start = performance.now()
  for (const s of strings) te.encode(s)
  console.log({ arrow, msPerEncode: (performance.now() - start) / 200 })
}

In a Solid app, render any component whose static template text contains a non-ASCII character next to a large amount of other markup, then profile renderToStream. Most of the encode time lands in encodeUtf8String.

Proposal

When the compiler builds SSR template strings, it can emit non-ASCII characters in static text and quoted attribute values as numeric character references (→ becomes →). Browsers decode these to the same characters, so the DOM, hydration and the client templates are unchanged. The work happens once, at build time. Tag internals, comments and <script>/<style> bodies have to keep their characters, because browsers do not decode references there.

This only helps when the non-ASCII characters come from templates. Non-ASCII in rendered data, such as a user's name, still makes the page two-byte.

Spike results

I tried this in the Rust compiler as a pass applied to each registered SSR template, with a small state machine for text, tags, quoted attributes, comments and raw-text elements. I then rebuilt the framework-tracker app with it:

The page without the arrow (/ssr-throughput) was unchanged at 2.13 ms. The rendered HTML was identical apart from the character references, the client bundle was byte-identical, and the page hydrated and navigated client-side in Chrome with no console output.

Two things are still open in the spike. The Babel plugin needs the same change: 54 compiler fixture tests currently fail, every diff being only a character reference, because their expected output comes from Babel. Attribute strings built for spread elements (ssrElement) take a different path and are not covered yet.

I can open a PR with the compiler change if this direction is wanted.