Re: #11828

This is a first pass at upgrading to the latest remark-parse version, based on micromark, for Markdown/MDX.

Current status

This is far from complete: 551 snapshots are failing on this branch (though some of those are due to harmless behavior changes; see Changes around escaping below). I'm hoping this will be a useful starting point for other contributors who can get this across the finish line.

Most of the work I've done here is getting Jest to run against all the ESM modules in the new packages, by running them through babel-jest. I've also replaced the custom plugins with remark- packages where possible, though in some cases these don't have all of the same behaviors (e.g. remark-frontmatter doesn't recognize this custom pragma) and may need to be replaced with new custom implementations.

Next steps

There are two in-house plugins, liquid and loose-items, that don't appear to have any remark- package equivalents, so someone will have to figure out how to reimplement those with micromark.

Once those are implemented, we need to take a careful look at the failing snapshots, decide which ones are blockers, and take steps to resolve them.

Finally, I've been focused on getting the unit tests running, but the build config will also need to be updated in order to get this branch running.

Changes around escaping

Many of the failing snapshots are caused by extra escaping. For example, the input

_ http://www.example.com:80/_a_/**

was previously converted to

_ http://www.example.com:80/_a_/\*\*

but on this branch comes out as

\_ http://www.example.com:80/_a_/\*\*

That is, the result is the same aside from the unmatched _ at the start being escaped.

That change may be a positive one? The current behavior is that the opening _ is escaped in the case where the input contains another _, e.g. _ http://www.example.com:80/_a*/* (the output is \_ http://www.example.com:80/_a*/*). So the new escaping behavior is more consistent, but noisier.