Skip to content

Feat: support map_from_arrays#1932

Merged
andygrove merged 6 commits intoapache:mainfrom
kazantsev-maksim:map_from_arrays
Jul 10, 2025
Merged

Feat: support map_from_arrays#1932
andygrove merged 6 commits intoapache:mainfrom
kazantsev-maksim:map_from_arrays

Conversation

@kazantsev-maksim
Copy link
Contributor

Which issue does this PR close?

Related to Epic: #1044
Spark: https://spark.apache.org/docs/latest/api/sql/index.html#map_from_arrays

Rationale for this change

Defined under Epic: #1044

What changes are included in this PR?

  1. maps.scala - added all already implement map funcs
  2. QueryPlanSerde.scala - map_from_arrays pattern matching case has been added and micro refactoring

How are these changes tested?

A new UT has been added.

@codecov-commenter
Copy link

codecov-commenter commented Jun 25, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 58.67%. Comparing base (f09f8af) to head (8c31b06).
Report is 297 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1932      +/-   ##
============================================
+ Coverage     56.12%   58.67%   +2.54%     
- Complexity      976     1145     +169     
============================================
  Files           119      133      +14     
  Lines         11743    13029    +1286     
  Branches       2251     2411     +160     
============================================
+ Hits           6591     7645    +1054     
- Misses         4012     4168     +156     
- Partials       1140     1216      +76     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kazantsev-maksim
Copy link
Contributor Author

Thanks @andygrove!


import org.apache.comet.testing.{DataGenOptions, ParquetGenerator}

class CometMapExpressionSuite extends CometTestBase {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new suite will need to be added in .github/workflows/pr_build_linux.yml and .github/workflows/pr_build_macos.yml before it can run in CI. I noticed that the CometBitwiseExpressionSuite suite also needs to be added.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed!

Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @kazantsev-maksim

@andygrove andygrove merged commit b79a301 into apache:main Jul 10, 2025
98 of 100 checks passed
@kazantsev-maksim kazantsev-maksim deleted the map_from_arrays branch September 28, 2025 14:20
coderfender pushed a commit to coderfender/datafusion-comet that referenced this pull request Dec 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants