Skip to content

feat(highlights): add 35 languages (120 total) - #1230

Merged
ije merged 5 commits into
mainfrom
highlights/120-languages
Oct 26, 2026
Merged

ije merged 5 commits into
mainfrom
highlights/120-languages

Conversation

@ije

@ije ije commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Add 35 lexers, picked by popularity (GitHub file counts, Stack Overflow questions, VS Code extension installs). Highlights now supports 120 languages.

New: ada, apache, apex, asciidoc, awk, bicep, blade, cobol, coffee, dotenv, erb, gherkin, gnuplot, haml, handlebars, haxe, http, ini, jinja, liquid, log, nginx, plsql, po, prisma, prolog, pug, qml, razor, rst, tcl, twig, verilog, vhdl, viml.

Key changes

  • Shared lexers: plsql uses sql.wat, apex uses java.wat, jinja/twig/liquid/handlebars share jinja.wat, blade/razor share template.wat.
  • Static memory: live-only buffers move to page 2. The keyword pool uses 14-bit offsets (16 KB). Stream checkpoints and byte sets get more room.
  • Live tokenizer copies only the saved part of the checkpoint region for each line.

Size

main PR
highlights.wasm 214,433 B 280,544 B
gzip 92,081 B 121,347 B

Live editing (bench-live)

Fixture Scenario main PR
large.ts (10k lines) eager init 4.49 ms 4.02 ms
large.ts (10k lines) template propagation 1.35 ms 0.87 ms
synthetic 100k lines eager init 36.9 ms 32.5 ms
synthetic 100k lines template propagation 29.2 ms 24.3 ms
unicode 10k lines eager init 3.31 ms 3.02 ms
unicode 10k lines template propagation 1.05 ms 0.65 ms

ref #1228

Add lexers for ada, apache, apex, asciidoc, awk, bicep, blade, cobol,
coffee, dotenv, erb, gherkin, gnuplot, haml, handlebars, haxe, http, ini,
jinja, liquid, log, nginx, plsql, po, prisma, prolog, pug, qml, razor, rst,
tcl, twig, verilog, vhdl, and viml, plus Shiki aliases for existing lexers.

Move live-only buffers to page 2 to make room for keyword tables, widen
keyword pool offsets to 14 bits, and copy only the saved part of the
checkpoint region per live line.
@vercel

vercel Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
pierre-diffshub Ready Ready Preview Oct 26, 2026 8:56am UTC
pierre-docs-diffs Ready Ready Preview Oct 26, 2026 8:56am UTC
pierre-docs-trees Ready Ready Preview Oct 26, 2026 8:56am UTC
pierrejs-diff-demo Ready Ready Preview Oct 26, 2026 8:56am UTC

Request Review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T02:14:12.796896Z ef22986 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ef22986082

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/highlights/src/live.wat Resolved
Comment thread packages/highlights/src/langs/ruby.wat Resolved
- live: store only the nonzero checkpoint range per line state, so later
  lexer windows no longer carry the zero windows before them
- ini: a blank line ends a backslash continuation in whole-buffer mode
- plsql: `q'` before a blank, tab, or line break is not a quote
- rst/asciidoc: non-ASCII letters block inline markup
- apex: a lowercase type word before an operator is a variable
- build: check the language name table against its region
Most test files compile the whole wasm module in a setup hook. On a loaded
CI runner those hooks passed Bun's 5 s default.

This branch was successfully deployed

4 active deployments
Preview – pierre-docs-diffs — 20dd6a08 Deployed Oct 26, 2026 by vercel[bot]
Preview – pierre-diffshub — 20dd6a08 Deployed Oct 26, 2026 by vercel[bot]
Preview – pierre-docs-trees — 20dd6a08 Deployed Oct 26, 2026 by vercel[bot]
Preview – pierrejs-diff-demo — 20dd6a08 Deployed Oct 26, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant