fix: exception during exception-handling when ast/CodeBlockNode is not set - #12766

For context, can you show an example of a meson.build that triggers this issue?

Also: please apply fixups to previous commits using git commit --amend and force pushing rather than creating multiple commits. You can also use git rebase -I origin/master and change the "pick" line for the fixup commit from "pick" to "f" to amend it after the fact.

For context, can you show an example of a meson.build that triggers this issue?

Also: please apply fixups to previous commits using git commit --amend and force pushing rather than creating multiple commits. You can also use git rebase -I origin/master and change the "pick" line for the fixup commit from "pick" to "f" to amend it after the fact.

I think I had a malformed meson.build file with "invisible characters" that tripped up the lexer - and then this follow-up-exception occured in vscode.

My feeling is that the bad character should be detected sooner, at the Lexer level.

Exactly. I'd still like to see an actual example meson.build file that reproduces the problem with sufficient fidelity that it can be checked into the repository as a test case -- not "a malformed file that tripped up the lexer". :)

(In fact we should have a test case regardless of whether we fix it in the AST interpreter or the lexer.)

Exactly. I'd still like to see an actual example meson.build file that reproduces the problem with sufficient fidelity that it can be checked into the repository as a test case -- not "a malformed file that tripped up the lexer". :)

(In fact we should have a test case regardless of whether we fix it in the AST interpreter or the lexer.)

I just recently figured out why python (in general, not just meson) had so much trouble on my machine.
system:

• windows (any 10+ should work, possibly 7+)
• vscode

steps to reproduce:

• add this setting to vscode: "files.encoding": "utf8bom" (which python calls encoding="utf-8-sig")
• edit a meson.build-file with vscode and safe it.
• run meson on that file

Python will try to open the file with utf-8 by default and fail to parse the BOM-"header"(?), tripping up the lexer and then the errors cascade from there.

expected behavior:

1. proper error-message from the lexer: finding invisible characters is near impossible if the tool doesn't help you.
   meson does not forward the full exception IIRC which makes this even harder. line-number and index would be really helpful.
2. no secondary error when self.ast is None (which I was trying to fix)
3. ideally: auto-attempting utf-8-sig import? maybe?

Opening build files using utf-8-sig encoding seems reasonable to me. This will still work with conventional utf-8 files, but will allow to silently drop the BOM when it is present.

As an aside, I'm reworking some of the state handling in the Interpreter, and I've changed the ast to use a dummy CodeBlockNode, which should fix some of these errors.

Also, Either using utf-8-sig or specifying that the format must be utf-8 no-bom and then adding some exception handling code for that seems appropriate. We could always do that with a wrapper function that sets the encoding properly and just use that instead of the raw open command

Python will try to open the file with utf-8 by default and fail to parse the BOM-"header"(?), tripping up the lexer and then the errors cascade from there.

Can you add a new test case:

test cases/failing/130 buildfile encoding/meson.build

containing this issue? That way the testsuite will detect this case and make sure it does not regress.

You can see how it is done in

test cases/failing/96 invalid option file/

for inspiration, but we want to check meson.build instead of meson_options.txt this time.