Skip to content

fix: requested character too large for encoding in chr function#552

Merged
andygrove merged 10 commits intoapache:mainfrom
vaibhawvipul:issue-480
Jun 14, 2024
Merged

fix: requested character too large for encoding in chr function#552
andygrove merged 10 commits intoapache:mainfrom
vaibhawvipul:issue-480

Conversation

@vaibhawvipul
Copy link
Contributor

Which issue does this PR close?

Closes #480 .

Rationale for this change

improves compatibility with spark

What changes are included in this PR?

  • added support for negative integers while encoding
  • also added support for integers larger than 256

How are these changes tested?

Test cases added, all the existing tests pass.

@vaibhawvipul vaibhawvipul marked this pull request as ready for review June 11, 2024 05:39
@vaibhawvipul
Copy link
Contributor Author

@andygrove this is ready for review.

@vaibhawvipul vaibhawvipul requested a review from andygrove June 11, 2024 18:02
@vaibhawvipul vaibhawvipul requested a review from andygrove June 12, 2024 01:41
@vaibhawvipul vaibhawvipul changed the title bug: requested character too large for encoding in chr function bug fix: requested character too large for encoding in chr function Jun 12, 2024
Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @vaibhawvipul

@andygrove andygrove changed the title bug fix: requested character too large for encoding in chr function fix: requested character too large for encoding in chr function Jun 12, 2024
None => {
exec_err!("requested character too large for encoding.")
.map(|integer| {
if integer < 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andygrove can we use unlikely (requires nightly) to indicate a less likely branch

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we are moving to stable?

@vaibhawvipul vaibhawvipul requested a review from viirya June 13, 2024 06:29
Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @vaibhawvipul. I think this addresses the feedback from @viirya so I will plan on merging this soon if there is no other feedback.

@viirya
Copy link
Member

viirya commented Jun 14, 2024

Thanks @vaibhawvipul

@andygrove andygrove merged commit 1a560c6 into apache:main Jun 14, 2024
@vaibhawvipul vaibhawvipul deleted the issue-480 branch June 14, 2024 15:21
himadripal pushed a commit to himadripal/datafusion-comet that referenced this pull request Sep 7, 2024
…he#552)

* adding support for negative integers and modulo checks for int above 256, also added test case

* fix test case to avoid cast integer overflow error

* Update core/src/execution/datafusion/expressions/scalar_funcs/chr.rs

Co-authored-by: Andy Grove <[email protected]>

* returning negative ints as empty string earlier

* adding scalar test and removing unreachable code

* Update core/src/execution/datafusion/expressions/scalar_funcs/chr.rs

Co-authored-by: Andy Grove <[email protected]>

* Update spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala

Co-authored-by: Andy Grove <[email protected]>

* add a test case where integer & 0xFF == 0

---------

Co-authored-by: Andy Grove <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: requested character too large for encoding in chr function

4 participants