fix: comments in loc.tokens hide parentheses from hasParens - #1438

Fixes #1423.

The bug

FastPath#getPrevToken and #getNextToken index loc.tokens directly. The babel, babel-ts, typescript and flow parsers put comment tokens into that array; esprima and acorn do not. So a comment sitting between a node and its enclosing parenthesis is returned instead of the parenthesis, hasParens() concludes the node is unparenthesised, and the printer emits a second pair.

This breaks the identity documented in the README on completely unmodified source:

const source = "const a = (\n  // c\n  1\n);";
recast.print(recast.parse(source, { parser })).code;
// babel, babel-ts, typescript, flow  ->  "const a = (\n  // c\n  (1)\n);"
// esprima, acorn                     ->  unchanged, correct

The issue as reported used a 60-line JSX and ternary example. Neither is required — the four lines above are enough, and the parser split is the whole story.

Token dump for that input under babel, where init.loc.start.token is 5:

0 const   1 a   2 =   3 (   4 CommentLine " c"   5 num "1"   6 )   7 ;   8 eof

getPrevToken returns index 4, the comment, rather than index 3, the (. Under esprima the comment is absent from tokens and the same lookahead lands on the parenthesis.

Both directions are affected, so trailing comments break too:

"const a = (\n  1\n  // c\n);"      // -> "const a = (\n  (1)\n  // c\n);"
"const a = (\n  1 /* c */\n);"      // -> "const a = (\n  (1) /* c */\n);"

The change

Both lookaheads now walk past comment tokens, and return null rather than a comment if that is all that remains. hasParens() already treats null as "no parentheses", so that path is unchanged — but neither function can now hand a caller something that is not syntax.

Comment tokens are identified by their string type (CommentLine / CommentBlock), which is how all four affected parsers emit them; syntactic tokens carry a token-type object instead.

Verification

Against main across 5,076 generated inputs — six parsers, twenty expression forms, six comment placements, eight enclosing contexts:

npm run mocha: 776 passing, up from 751. npm run lint: clean.

Tests

Added to test/comments.ts, which already runs each case against acorn, babel, esprima, flow and typescript. Five placements: leading line, trailing line, leading block, trailing block, and both sides.

Worth noting what happens with the fix reverted but the tests kept: 15 of the 25 instances fail, and the 10 that pass are exactly the acorn and esprima ones. The suite demonstrates the parser split on its own.

Related

#1129 has been open since 2022 and also touches lib/fast-path.ts, for #575. It changes needsParens() rather than the token lookaheads. I applied its hunk to current main and confirmed it does not address this issue, so the two should not conflict.