Add dictionary support to unhex and test dictionary and scalar cases #477

Activity

1. kazuyukitanimura

   added

   bugSomething isn't workingSomething isn't working

   on May 27, 2024
2. andygrove

   added

   good first issueGood for newcomersGood for newcomers

   on Jun 16, 2025
3. kination commented on Jul 29, 2025

   

   kination

   on Jul 29, 2025

   Contributor

   More actions

   @kazuyukitanimura sorry for question.
   Is it meaning unhex cannot decode correctly if value is dict? For example...

   SELECT decode(unhex('<source>'), 'UTF-8'); 
   

   it works well if source hex data is String such as Spark, but it fails if it is { "target": "spark" }
   -> Am I understanding correct?
4. kazuyukitanimura commented on Jul 31, 2025

   

   kazuyukitanimura

   on Jul 31, 2025

   Last edited by kazuyukitanimura

   ContributorAuthor

   More actions

   Thanks @kination
   In the unhex test https://github.com/apache/datafusion-comet/blob/main/spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala#L1433
   We only have limited test cases.
   You can use makeParquetFileAllPrimitiveTypes to create test dictionary values and some random values

   The test case with dictionary may fail because I do not see dictionary handling at https://github.com/apache/datafusion-comet/blob/main/native/spark-expr/src/math_funcs/unhex.rs#L67

   It is possible the tests with dictionary still passes if we are unpacking in advance. But I forgot we are doing so
5. coderfender

   added a commit that references this issue on Dec 13, 2025

   build: Use Iceberg branch apple-1.5.0.x for 3.4 (i.e. 3.4.3) (apache#477

   ...

   421a981
6. andygrove

   added

   priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

   area:expressionsExpression evaluationExpression evaluation

   on Mar 27, 2026
7. andygrove

   closed this as completedin #4222on May 6, 2026