Skip to content

feat[lang]: add native hex string literals#4271

Merged
charles-cooper merged 24 commits intovyperlang:masterfrom
tserg:feat/native_hex
Oct 11, 2024
Merged

feat[lang]: add native hex string literals#4271
charles-cooper merged 24 commits intovyperlang:masterfrom
tserg:feat/native_hex

Conversation

@tserg
Copy link
Copy Markdown
Contributor

@tserg tserg commented Oct 4, 2024

What I did

Resolves #4206

How I did it

Add a custom parser to watch for STRING tokens whenever there is a NAME tokens of "x" value, store the location of these hex literals, and transform them to byte values during transformation of the python ast to vyper ast.

How to verify it

See new tests.

Commit message

This commit adds support for custom hex literals like `x"a8b9"`. this
hex literal is equivalent to the bytestring `b"\xa8\xb9"`. this syntax
is a break from Python, which does not have this type of literal, but
it is helpful in the smart contract space because often literals are
copy-pasted around as hex.

the approach taken in this PR is to add another custom sub-parser in
`pre_parse.py` which keeps track of where the hex literals are in
the original source code. it additionally refactors the signature of
`pre_parse()` to return a `PreParseResult`, which bundles all the data
structures returned by `pre_parse()` together.

Description for the changelog

Cute Animal Picture

Put a link to a cute animal picture inside the parenthesis-->

@tserg tserg changed the title feat: native hex string literals feat[lang]: native hex string literals Oct 4, 2024


class ForParserState(enum.Enum):
class CustomParserState(enum.Enum):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe just ParserState

@charles-cooper
Copy link
Copy Markdown
Member

please add docs!


byte_val = bytes.fromhex(node.value)

node.ast_type = "Bytes"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am not a huge fan of fusing it into the Bytes type, since we lose information in the AST. let's make a new VyperNode AST type, maybe XBytes or HexBytes?

return ast_dict

@property
def s(self):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wonder what would happen if we removed all of these 🤔

out of scope for this PR but we should check if that is still needed, since it was basically originally introduced for backwards compatibility

Comment on lines +345 to +347
node.ast_type = self._pre_parse_result.modification_offsets[
(node.lineno, node.col_offset)
]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's try to avoid the multi-line formatting

Suggested change
node.ast_type = self._pre_parse_result.modification_offsets[
(node.lineno, node.col_offset)
]
key = (node.lineno, node.col_offset)
node.ast_type = self._pre_parse_result.modification_offsets[key]

return True


class NativeHexParser:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
class NativeHexParser:
class HexStringParser:

Copy link
Copy Markdown
Member

@charles-cooper charles-cooper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good. just a couple ideas about naming, and let's try to keep multiline statements to a minimum

@charles-cooper charles-cooper changed the title feat[lang]: native hex string literals feat[lang]: add native hex string literals Oct 11, 2024
@charles-cooper charles-cooper enabled auto-merge (squash) October 11, 2024 21:32
@charles-cooper charles-cooper merged commit 212ff59 into vyperlang:master Oct 11, 2024
@tserg tserg deleted the feat/native_hex branch October 12, 2024 01:20
pcaversaccio added a commit to pcaversaccio/snekmate that referenced this pull request Oct 18, 2024
### 🕓 Changelog

Vyper has introduced support for native hex string literals, such as
`x"a1b9"` (see PR
[#4271](vyperlang/vyper#4271)). This update
refactors the 🐍 snekmate contracts—`base64` and `message_hash_utils`—to
take advantage of the new syntax. However, the `internal` function
`_to_eth_signed_message_hash` remains unchanged, as its current
implementation with `\x19` provides better clarity. Moreover, the
`external` function `eip712Domain` in the `eip712_domain_separator`
contract has been simplified to utilise a `bytes1` literal.
Additionally, all submodules have been updated to their latest commits,
and the latest Vyper `master` commit is now once again applied in all
`venom`-based tests, following the resolution of issue
[#4288](vyperlang/vyper#4288) through this
[commit](vyperlang/vyper@b3ea663).

---------

Signed-off-by: Pascal Marco Caversaccio <[email protected]>
willbrown84 added a commit to willbrown84/snekmate that referenced this pull request Sep 23, 2025
### 🕓 Changelog

Vyper has introduced support for native hex string literals, such as
`x"a1b9"` (see PR
[#4271](vyperlang/vyper#4271)). This update
refactors the 🐍 snekmate contracts—`base64` and `message_hash_utils`—to
take advantage of the new syntax. However, the `internal` function
`_to_eth_signed_message_hash` remains unchanged, as its current
implementation with `\x19` provides better clarity. Moreover, the
`external` function `eip712Domain` in the `eip712_domain_separator`
contract has been simplified to utilise a `bytes1` literal.
Additionally, all submodules have been updated to their latest commits,
and the latest Vyper `master` commit is now once again applied in all
`venom`-based tests, following the resolution of issue
[#4288](vyperlang/vyper#4288) through this
[commit](vyperlang/vyper@b3ea663).

---------

Signed-off-by: Pascal Marco Caversaccio <[email protected]>
void-rider5560p added a commit to void-rider5560p/snekmate that referenced this pull request Sep 28, 2025
### 🕓 Changelog

Vyper has introduced support for native hex string literals, such as
`x"a1b9"` (see PR
[#4271](vyperlang/vyper#4271)). This update
refactors the 🐍 snekmate contracts—`base64` and `message_hash_utils`—to
take advantage of the new syntax. However, the `internal` function
`_to_eth_signed_message_hash` remains unchanged, as its current
implementation with `\x19` provides better clarity. Moreover, the
`external` function `eip712Domain` in the `eip712_domain_separator`
contract has been simplified to utilise a `bytes1` literal.
Additionally, all submodules have been updated to their latest commits,
and the latest Vyper `master` commit is now once again applied in all
`venom`-based tests, following the resolution of issue
[#4288](vyperlang/vyper#4288) through this
[commit](vyperlang/vyper@b3ea663).

---------

Signed-off-by: Pascal Marco Caversaccio <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

VIP: native hex string literals

3 participants